4 of 20

The International ACM Symposium on High-Performance Parallel and Distributed Computing (HPDC) will commemorate its first 20 years with a special proceeding containing the best 20 papers. Rich Wolski, and Steve Fitzgerald, respectively founder and CTO, and VP of Technical Services at Eucalyptus Systems, will each have 2 papers published in that proceeding. Rich will have Forecasting Network Performance To Support Dynamic Scheduling Using The Network Weather Service (NWS) and Scheduling from the Perspective of the Application (AppLeS) while Steve will have Grid Information Services for Distributed Resource Sharing and Application experiences with the Globus toolkit. That is 20% of the papers!

I have been fortunate enough to have been working with them since the mid '90, back when a cloud on a whiteboard meant Internet, and Grid Computing was the pinnacle of Distribute Computing research. In addition to be excellent scientists, Rich and Steve have also been role models, teaching the new generations for more than 15 years. It's hardly a coincidence then, that Rich and Steve are now pushing the boundaries of  knowledge in Cloud Computing with Eucalyptus.

It's hard to overestimate the influence that Rich and Steve have within our organization. For example, in the latest Eucalyptus 3, I can find Rich's fingerprints in the overall architecture, in the attentions to the internal protocols, and Steve's touch in the identity integration (LDAP) and management (EIAM).  They are mentors and role model to all eucalyptoids, for their determination, insights, and skillful leadership each with his own personable yet efficient style.

Edit: check out the full list.

PyCon 2012

We (cpt_yesterday, gholms and obino) left Silicon Beach (aka Santa Barbara home of Eucalyptus) on Thursday and drove up to Santa Clara for my first PyCon.

Driving from Silicon Beach to Silicon Valley

I was a bit leery about dinner (just few week ago cpt_yesterday and I got food poisoned in Sunnyvale), but all went well and it was great to see Mitch.

Setting up the booth

We set up the booth on the morning, just before the keynotes. The booth has been a good meeting point throughout the conference to talk about Eucalyptus, get a bit of rest, or just grab one of our t-shirts. It's always rewarding to talk about how Eucalyptus is used in production or to receive thumbs up from people walking by.

Mitch and cpt_yesterday

This was my first PyCon, and been a Python n00b myself (my first python program is few weeks old), it was a bit intimidating to be around such a crowd. Everyone was very helpful and engaging and willing to tolerate newbie: a very nice experience.

  

gholms likes PyCon

We got a bit distracted from the conference by PG&E: they told us on Friday night that an urgent safety fix was needed and Saturday they will pull the power all around our HQ. Like all good DevOps we spent quite a bit of time to prepare for the outage, coordinating with the team back at HQ. Our core services runs on our public Eucalyptus cloud at CoreSite (read more about the ECC for more info on our setup) but we have an internal cloud we use at HQ, and we wanted to be sure all these internal services were shut down nicely.

The stars of the conference

And of course I needed a picture of the stars of PyCon 2012.

First steps with Eutester

In one of my previous post I mention our mantra: "Listen to our community and deliver quality software". To deliver quality software, it is necessary to have the QA process as a first class citizen. Our QA team created quite a spectacular infrastructure to test all sort of combinations and configurations automatically (distro, architectures, versions, hypervisors, networking, images etc ...) to guarantee our users, regression-free releases. They also managed to have fun in the process (check out Pigeons on a Euca).

Eutester is the latest brainchild of our QA team, and this blog is about my experience writing a test using it. Eutester is a framework to create automatic tests against a Eucalyptus installations (or any cloud following the AWS API for that matter).

 These are the ingredients I needed to bake my first test:

• a really annoying bug
• a Eucalyptus cloud to test it against
• the Eutester framework
• basic python knowledge helps although it is not necessary

Let's start with the main ingredient: a really annoying bug. I picked lp:737335, annoying and small enough for me to tackle. The issue reported is that the launch permissions as set by euca-modify-image-attribute are not respected in Eucalyptus 2.0. The bug happened also to have a proposed fix.

 Next: reproduce the issue on a Eucalyptus cloud of the desired version. This bug is easy to reproduce: I just need to upload an image, change the launch permission to remove all user, then try to launch it: if the instance starts the issue is present. Since I wanted to test the proposed patch, I did installed Eucalyptus 2.0.3, compiling it from source. Lester also tested it against the development branch of Eucalyptus 3.1 (check Andy's blog to work with the devel branch, and Greg's blog for more info on 3.next) and confirmed that only 2.0 is affected.

 I then downloaded Eutester, installed it locally on my client machine and started to hack the test. I used virtualenv to ensure a clean, reproducible environment to work with. My first attempt of the test successfully reproduced the issue, so I gingerly submitted it for integration. The QA team is still laughing at my python skills ... They kindly reworked the test and it is now sitting into the Eutester repository. With the new test added to a sequence in our QA system I am now guaranteed that all future version of Eucalyptus will be tested for regression against this issue.

 Onward to the patch for 2.0. I created a branch with the fix, and proposed it for merging. The QA system caught my sloppy programming at the first attempt (typo), but the second version was the win: QA passed with flying colors and the branch was eventually merged into the top of the Eucalyptus 2.0 branch.

Quite an accomplishment here: my first python script, my first Eutester test and no more lp:737335, just the perfect happy ending for this blog.