Updated Vagrantfile for bin/run-tests-vagrant script

[dpb587]

First, I cleaned and updated the Vagrantfile. I verified it with both aws and virtualbox.

Second, I added a bin/run-tests-vagrant script. I envision a Jenkins build script with:

Environment Variables:

• AWS_KEYPAIR_NAME - I created a new jenkins-slave-20140218a key and added it to eu-west-1 and Jenkins' configuration file plugin
• AWS_PRIVATE_KEY_PATH - auto-generated by the configuration file plugin
• AWS_VAGRANT_USE_IAM=true - for vagrant creating the instance
• AWS_VAGRANT_IAM_PROFILE_NAME - tests require the cloudformation:ValidateTemplate permission

Script:

chmod 700 $AWS_PRIVATE_KEY_PATH
./bin/run-tests-vagrant aws destroy

---

This is pull request is related to the build issues of #303, #300

[dpb587]

@sopel, do you have thoughts on this approach and whether we could use vagrant on jenkins somewhere to better use it?

[sopel]

Sorry for the late reply - I'm currently in an ongoing state of slight confusion regarding the state of our build, what I'd like it to evolve to in the mid and long term, and how to get from the former to the latter in an agile and non disruptive way ;)

That being said, I very much like the cleanup aspect of this refactoring and the direction you are heading at, so :+1: from me.

[sopel]

Not entirely sure whether I get you correctly here, but regarding Vagrant on Jenkins I'm similarly undecided currently btw. - Vagrant is fantastic to get a cross platform unified development environment going, both locally and on AWS, and IMHO every repository should (still) offer a Vagrantfile to get everyone started with the project quickly.

However, the advent of Docker greatness kind of yields every non Windows Vagrant machine to basically be a Docker container host (too, for me at least), with the actual stuff being pushed further down into a Docker image in turn, as you just demonstrated for the scaling job.

• I realize that you haven't done that here yet, which might be the context of this discussion actually?

Now, with that in place, why should Jenkins (when running in the cloud, i.e. our team's Jenkins clusters) be concerned with Vagrant at all, when it can also just facilitate Docker right away (with all the benefits, most particularly much faster provisioning time in comparison to Vagrant)?

[sopel]

Put another way, I think containerization trumps virtualization for build management (too), even though the former might be required to enable the latter for cross platform usage still (i.e. to get a containerized build running on Windows for example), or to simply run a containerized build in the cloud via the AWS provider indeed (as you just prepared here).

[sopel]

So in hindsight, after rereading your proposal, I very much agree with the concept of spinning a dedicated build environment up and down per job (and I guess @mrdavidlaing will too, insofar he has promoted that for a while as well - we have also evaluated Strider CD already in this regard, see https://github.com/cityindex/labs-operations/issues/128), but I think basing this on Vagrant rather than Docker might be unnecessarily costly (time wise in particular, but cost wise on the side, insofar a single EC2 instance could serve many such Docker based build jobs during an instance hour)?!

[dpb587]

I agree with all your thoughts. I wasn't sure about introducing Docker at the root level at this point and whether everyone was in agreement on Docker's usage. Ultimately I think I'd like to see each of our services to be fully independent Docker images (and repos?).

In the meantime I wanted to improve my testing workflow with a one-liner test command to test my widespread spacing changes, and having been on a Jenkins mindset I thought it could be reused there.

If/when we confirm Docker is okay for this, I was planning on a bin/run-tests-docker sort of script and moving the actual testing part of bin/run-tests-vagrant to bin/run-tests.

I'm certainly open to further discussion about this to get better ideas and particularly to make sure we're all on the same page on where we want our builds to go and how to get there. In the meantime, I don't think this particular PR goes against directly anything we'll be changing for Docker if it's in the future.

[sopel]

Cool, my initial :+1: has been holding true anyway, so :shipit: