A Developer Cloud

The title of this blog says it all: in this blog I will detail how I created my own private Eucalyptus cloud on my laptop, using VMs, bridge and iptables, and of course Silvereye. The instructions may be specific of my laptop, which runs Debian Sid. The relevant specs of the laptop (a Lenovo x220t to be precise) are: 8GB of RAM, CPU is INTEL i5-2520M, and a 160GB SSD. Let me repeat: this is a developer cloud setup, which means that it will do the testing I need to do, but it doesn't run any production environment, nor is meant to. Dustin created Cloud on a Stick and this work was inspired by Dustin's excellent work with UEC.

The idea is simple: create 2 VMs (respectively front-end and node controller), attach their NIC to a local bridge, use NAT for when the VMs need external connectivity, install Eucalyptus 3-devel on them, and play with it. Few reasons for the above setup:

• 2 VMs because I want to use the MANAGED or MANAGED-NOVLAN networking modes (there is no HTML documentation yet, so get the manuals for more information on 3.1 networks modes) to take full advantage of security groups and elastic IPs;
• a local bridge (with no physical device attached to it) because I keep changing networks between wireless and wired (and no network for my coffee shop breaks), and because most of the time I don't need the cloud to talk to the outside world, so NATting is sufficient for me.

Let's start setting up the virtualization bits. I use KVM on my laptop, and I will take advantage of the nested virtuatlization capabilities of the CPU I have. The kvm_intel module do not enable it by default, so I added a x220t.conf file to /etc/modprobe.d with the following

       options kvm_intel nested=1

and reloaded the module (rmmod kvm_intel and then modprobe kvm_intel). My old laptop had an AMD Turion and nested virtualization was on by default: if you are attempting to replicate this YMMV.

Next, let's setup the bridge to nowhere. I could have used the default bridge setup by libvirt (called virbr0), but I found it easier to setup my own own. I followed the QEMU Debian wiki to setup the tap devices I needed. Note how only tap0 and tap1 are enslaved to the bridge. The complete entry (excerpt from /etc/network/interfaces) is

 auto br0

 iface br0 inet static

address 172.17.0.1

netmask 255.255.255.0

pre-up ip tuntap add dev tap0 mode tap user graziano

pre-up ip link set tap0 up

pre-up ip tuntap add dev tap1 mode tap user graziano

pre-up ip link set tap1 up

bridge_ports tap0 tap1

bridge_stp off

bridge_maxwait 0

bridge_fd      0

post-down ip link set tap0 down

post-down ip tuntap del dev tap0 mode tap

post-down ip link set tap1 down

post-down ip tuntap del dev tap1 mode tap

When I need my VMs to reach the outside world, I ensure that ip forward is enabled

          echo 1 > /proc/sys/net/ipv4/ip_forward

and I use the simplest NAT rules I found:

     /sbin/iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE

     /sbin/iptables -A FORWARD -i wlan0 -o br0 -m state   --state RELATED,ESTABLISHED -j ACCEPT

     /sbin/iptables -A FORWARD -i br0 -o wlan0 -j ACCEPT

The above works when I'm connected with the wireless card; I substitute wlan0 with  eth0 otherwise. I didn't add any rule to the default network configuration since I don't allow the cloud to be online all the time.

I need the last few pieces, before I can start the instances. I already have the Silvereye iso, so I need to create a file big enough to hold the Eucalyptus component. I don't have too much space on the disk, so I settled for 10GB and 15GB respectively for the front-end (FE) and node controller (NC). I did use the following command

        dd if=/dev/zero of=nc.img count=1 bs=1G seek=10

to create both fe.img (for the front-end) and nc.img (for the node controller).

Last piece is to create the libvirt xml configuration for the 2 instances. Here is the front-end one:

<domain type='kvm'>

  <name>frontend</name>

  <memory unit="GiB">2</memory>

  <description>Front End</description>

  <cpu match='exact'>

    <model>core2duo</model>

    <feature policy='require' name='vmx'/>

  </cpu>

  <os>

    <type arch="x86_64">hvm</type>

    <boot dev='cdrom'/>

  </os>

  <features>

    <acpi/>

  </features>

  <clock sync="localtime"/>

  <devices>

    <emulator>/usr/bin/kvm</emulator>

    <disk type='file' device='disk'>

      <source file='/home/graziano/Prog/ImagesAndISOs/fe.img'/>

      <target dev='vda' bus='virtio'/>

    </disk>

    <disk type='file' device='cdrom'>

      <source file='/home/graziano/silvereye.1337754931.857206.iso'/>

      <target dev='hdc'/>

    </disk>

    <interface type='ethernet'>

      <target dev="tap0" />

      <mac address='24:42:53:21:52:45'/>

    </interface>

    <graphics type='vnc' port='-1'/>

  </devices>

</domain>

note how the flag vmx has been forced. vmx is the flag indicating hardware virtualization for INTEL processors: AMD's one is svn. The NC configuration file is very similar: the network device is tap1,  I changed the MAC address, VM name, and the file backing the disk . I didn't even need to comment out the <boot> flag after installation, since Silvereye boots from local disk by default.

My machine was already setup to run instance with my username (you can check the connection to libvirt with virsh list), so I can start the FE with 

        virsh create frontend.xml

and similarly the NC using node.xml. The first time I booted them, I followed the steps in my previous blog on Silvereye (virsh vncdisplay is your friend when you have multiple VMs on VNC). I did ensure that the VMs were configured to use static IPs assignments (I used 172.17.0.2 for the frontend and 172.17.0.3 for the NC). Because I wanted to change the network configuration and hostname ahead of time I didn't run the Silvereye script at the first login, but I run them afterwards (the scripts are in /usr/local/sbin in the installed VM).

Few extra tips. I did install the NC few times: make sure you remove the /root/.ssh/known_hosts when re-registering the NC if you keep the same IP between installs. If you want to play with different image sizes make sure you have enough disk space on the NC (few re-installs were needed for me to settle on a proper image size/NC disk size and I wish I could use a much bigger disk for it). After the first time installation, I prefer not to use VNC to connect to the VMs, so I added logic for the console (just before the VNC line)



    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>



and in the instances I add to add console=ttyS0 to the boot line (in /etc/default/grub for Debian and in /boot/grub/grub.conf for CentOS). After a VM reboot I can login with 


    virsh console frontend

Finally, these are not  instances: if you modify them, delete things or make them unusable, well, they are gone. The good news is that they are very easy to re-create them.