Integration environment setup with Vagrant

A while ago I was tasked with building an integration environment for a service so that product owners and other stakeholders could check out the new versions before publishing to production and developers could test the code outside their laptops. As it often happens in these kinds of stories, we didn’t have much time to spend for this task.

The starting point was roughly as follows:

  • A recently acquired legacy system without existing integration environment.
  • Vagrant and Chef already in use for development and Chef alone for production (all configured by us since the previous developers didn’t use them).
  • The configuration system of the app is painful to use if you want to create additional environments. Basically a third environment would have needed either a lot of refactoring and testing or duplicating hundreds of lines of PHP.

Ideas for solution

1. Rent servers and go through the pain of configuring a third environment

As mentioned before, we have Chef, so the setup of the infrastructure is not a problem. It’s the actual PHP-app and it’s legacy configuration system that brings in the pain. This was pretty much a no-go from the beginning with the costs of renting new machines and the time needed to get it all running.

2. Build a pay-as-you-go integration system atop Amazon infrastructure

Sounded good, but would have needed lots of configuration and research (at the moment Amazon’s services are not well know in our team, situation which will change soon). This, however is a route which we’ll research later.

3. Rent one server, create a single chef-role that installs the whole system

Still requires a lot of configuration for the PHP-app and doesn’t feel “clean” as it doesn’t even remotely model the production environment which has multiple machines with clear separation of roles

4. Recycle what you already have

We use Vagrant and we have Chef-cookbooks for building our dev- and production-environments. Why not benefit from work that has already been done?

And the winner is?

Number 4, obviously. Tight schedule and limited budget left no other choices + it seemed like a fun idea to recycle the same Vagrant machines for integration environment.

Setting it up

A quick and dirty tutorial on how we setup the environment.

Either buy, rent or steal a server somewhere with high enough specs to run Vagrant and Virtual Box with couple virtual machines. Optionally buy a domain and point it to your new integration server. We did this because we wanted to model the integration server more closely with production and also for non-techies to have and easy to remember address rather than ip-address (or setting up hosts-files for everyone).

Install Ubuntu (or distro of your own preference, we though, like Ubuntu). Download the latest Vagrant package straight from their own site (recommended as the Ubuntu repos - especially 12.04 LTS - have very old Vagrant and Ruby package versions that don’t work at all).

Install the Vagrant package
sudo dpkg -i vagrant_[version].deb
Configure nginx on the host to act as a proxy
# put this to your sites-enabled -folder (eg. vagrant-proxy.conf)
server {

  server_name *.example.com;

  # proxy configuration
  proxy_set_header Host $host;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Original-Source-IP $remote_addr;

  # if you use SSL
  proxy_set_header X-SSL-Cipher $ssl_cipher;
  proxy_set_header X-SSL-Protocol $ssl_protocol;

  proxy_read_timeout 15s;

  client_max_body_size 10M;

  # proxy everything to the front-end machine running with a private IP
  location / {
    proxy_pass http://192.168.33.20;# change this
  }

}
Fire up vagrant
vagrant up

After the Vagrant virtual machine(s) are up, secure your new server with appropriate firewall settings and test that everything works okay. Done, have a nice cup of tea, go do something else, call your wife and kids that you’ll be home early.

Misc. gotchas & best practises

  • Consider creating a separate user account for running the Vagrant machines.
  • You might need to setup port forwardings from the host machine to the Vagrant machines - if you wish to eg. access the test database from SequelPro - this is, however quite easy to configure in Vagrant with just one line.

Afterthoughts

The concept of using Vagrant to build a throwaway cloud for housing an integration environment worked smoothly and it was quite quick to take into use. The biggest challenges we encountered were related to the way that the configuration system of the actual PHP app was implemented. But in the end, the amount of work to get this running was much less than it would have taken in the other scenarios outlined earlier. And if you look at this from the point of time and money consumption, this was a really efficient solution (especially as we had already invested time on learning Vagrant and Chef).

Overall, I’m really happy with the results and even more happy that we made the decision of going with Chef+Vagrant early on when we acquired the system and moved it to our server hall. That decision has saved countless hours of work already (I don’t even want to know what this would have taken in the olden days).