As promised, I’ll explain today what is the ghost RAM feature and how it can be used to decrease memory usage of the virtual router instances. First, it is important to understand how the virtual RAM (vRAM) works and how the IOS is loaded into it at startup. Depending on your dynamips/dynagen settings, the vRAM is simulated either from a memory-mapped file, either directly from host memory (with the well-known “malloc()” function). When a virtual instance is started, the Cisco IOS image is loaded into the vRAM.
The problem is that when you start many instances with the same image, the code is loaded and duplicated for each instance (since each instance has its own vRAM). The ghost RAM feature is a solution that helps to decrease memory usage by sharing the IOS image between different instances (technically, it relies on mmap() and Copy-On-Write mechanism to accomplish this). So, if you have an uncompressed IOS image of 50 Mb, and you start 5 instances, you should have a gain of 4 x 50 = 200 Mb.
Let’s take a simple example. The first step is to generate the ghost RAM file, which will contain the IOS image code shared by all instances. Use only uncompressed IOS images for that! (if you use compressed images, each instance will uncompress it for its own usage, and the result will not be shared).
chris@portchris$ ./dynamips -g ghost_ram image.bin
Cisco 7200 Simulation Platform (version 0.2.6-RC4-x86)
Copyright (c) 2005,2006 Christophe Fillot.
Build date: Nov 15 2006 23:05:16
IOS image file: image.bin
CPU0: carved JIT exec zone of 64 Mb into 2048 pages of 32 Kb.
NVRAM is empty, setting config register to 0×2142
C7200 instance ‘default’ (id 0):
VM Status : 0
RAM size : 256 Mb
IOMEM size : 0 Mb
NVRAM size : 128 Kb
NPE model : npe-200
Midplane : vxr
IOS image : image.bin
Loading ELF file ‘image.bin’…
ELF entry point: 0×80008000
C7200 ‘default’: starting simulation (CPU0 PC=0xffffffffbfc00000), JIT enabled.
Shutdown in progress…
Shutdown completed.
The instance is started with “-g” parameter to generate a file called “ghost_ram”. Once started, the instance stops, it is normal.
Now, we can start instances that will use this “ghost_ram” file, and share the IOS image loaded into it:
chris@portchris$ ./dynamips -i 0 -G ghost_ram image.bin
Cisco 7200 Simulation Platform (version 0.2.6-RC4-x86)
Copyright (c) 2005,2006 Christophe Fillot.
Build date: Nov 15 2006 23:05:16
Instance ID set to 0.
IOS image file: image.bin
CPU0: carved JIT exec zone of 64 Mb into 2048 pages of 32 Kb.
NVRAM is empty, setting config register to 0×2142
C7200 instance ‘default’ (id 0):
VM Status : 0
RAM size : 256 Mb
IOMEM size : 0 Mb
NVRAM size : 128 Kb
NPE model : npe-200
Midplane : vxr
IOS image : image.bin
Loading ELF file ‘image.bin’…
ELF loading skipped, using a ghost RAM file.
ELF entry point: 0×80008000
C7200 ‘default’: starting simulation (CPU0 PC=0xffffffffbfc00000), JIT enabled.
ROMMON emulation microcode.
Launching IOS image at 0×80008000…
[…]
Each instance will use the “ghost_ram” file as a “base”. Each time a page of this mapped file is to be modified (by a write operation of the emulated CPU), a local copy of it is automatically generated by the operating system. Since an IOS image mainly consists in code and some static data never modified, this can be shared easily.
Greg is currently adding support for this feature in Dynagen. Just one thing to remember: set the same IOS image and the RAM parameters when you start an instance using a ghost RAM file.