Switch from virtualbox to qemu in deployment system

[cwebber]

Virtualbox apparently has a non-free bios compiler dependency (wtf) and we're going for free as in freedom / open as in totally open, so! Let's see if we can replace virtualbox with qemu in our ansible deployment setup.

[cwebber]

There are a couple of routes:

• Switch from vagrant to a simple homerolled script (vagrant doesn't do that much, really)
• Use a vagrant plugin which supports qemu... this appears to be the best route.

Some resources:

• On creating and using qemu images
• From that page, a link to some prebuilt Debian images, by a Debian developer (Fairly out of date, but should be useful)
• Other testing-ish images

[cwebber]

Obviously we want to have reproducible builds of qemu images, or have the user build their own. I'll settle for a prebuilt image for testing first though.

[kfogel]

Yay! (Just watching and cheering; I trust your judgement about the right route to go here...)

[cwebber]

I got qemu working. That was easy, pleasant, and interesting.

However, getting Vagrant to work with qemu has turned out to be a real challenge. Here's what I ran into:

• Requires installing plugins through Vagrant itself rather than apt-get. Annoying, but not really a showstopper.
• Requires running some commands twice when installing said plugins because they fail the first time and work the second. Annoying/strange (but not the first time I've seen that happen with a language's package manager... :P)
• There's no nice way to create a "vagrant box" from a qemu image you have. You are stuck with two options:
  • Construct the box (a tarball with some metadata) manually using your qemu image and hand-editing some xml and json files. YUCK! Here is what that looks like, and I would not want to ask users when constructing their own box to make such an XML file by hand. That's not something we can ask.
  • Mutate an existing virtualbox or etc box using vagrant-mutate... but if we have to use Virtualbox to make the box, then are we really removing our Virtualbox depenency?
• To top that all off, I decided to see if I could get vagrant-kvm to work at all, so I converted the virtualbox based vagrant box with vagrant-mutate over to a kvm provider box, and then tried using that with vagrant-kvm... and it failed anyway with an extremely unhelpful message: "System command which redir > /dev/null returned with error code false." Thanks, buddy.

Vagrant doesn't do quite that much anyway. One thing that's nice about qemu images: they're just files you can copy around very easily. I think at this point we should ditch Vagrant... I have a much simpler suggestion, but I am going to write it up as a separate comment.

[kfogel]

I'm sold -- ditching Vagrant is fine. Looking forward to your upcoming suggestion comment!

[cwebber]

Here's my much simpler suggestion. We can start with something small and grow it a bit if needed.

For the whole "local virtual machine for development that you can setup and tear down", why not KISS? How about a simple makefile. At its very simplest:

# Install "base" vm if needed, copy over one for hacking, turn it on, and provision it
make
# turn it off
make halt
# Destroy it
make destroy

The default "make" target automatically will run some other targets, that can of course be run individually if desired::

# first-time setup of *base* virtual machine
$ make base
# make a "copy" of the vm for local hacking... copies the base file
$ make vm
# turn on the vm
$ make up
# provision the vm (doesn't provision the live server, we can use normal ansible commands for that)
$ make provision

(We could use a filename like "vm.make" and do "make -f vm.make" if we want to be clear about it.)

Karl, what do you think of this? We can eventually iterate to something more "sophisticated" if needed, but I think this will be fine. Again, Vagrant does not do that much.

If we figure this out nicely, it could be the basis for future OTS stuff too.

[cwebber]

Sketching things out:

.PHONY: up server-is-up vm base provision halt destroy

###################
# Default locations
###################

PIDFILE=.vm/running.pid
BASE_IMAGE=.vm/base-image.qcow2
DEV_IMAGE=.vm/dev.qcow2

######################################################################
# Targets
######################################################################

# Provision
# =========

# Turn on (if needed) and provision the server

provision: up
    ./deploy/scripts/provision.sh

# Spin up
# =======

up: vm $PIDFILE

# This is where we really turn it on
$PIDFILE:
    ./deploy/scripts/run_vm.sh

# Create the base image
# =====================

base: $BASE_IMAGE

# This is where we really make the base image
$BASE_IMAGE:
    ./ansible/scripts/make_base_image.sh

# Create the user's dev VM
# ========================

vm: base $DEV_IMAGE

$DEV_IMAGE:
    cp $BASE_IMAGE $DEV_IMAGE

# Turning stuff off and destroying
# ================================

halt:
    ./deploy/scripts/halt_if_running.sh

destroy: halt
    -rm -f $DEV_IMAGE

destroy-base: halt
    -rm -f $BASE_IMAGE

[cwebber]

I've started a branch with these changes: https://github.com/OpenTechStrategies/lisc-ttm/tree/qemu

There are some errors in the "sketched out" makefile above, but I'm fixing things up as I go.

[kfogel]

+1 on this solution. Is there a way to specify a custom name for the environment being made? Or is that just "the directory in which one is doing all this"?

[kfogel]

By the way, if this is not TTM-specific, let's break it out into its own open-source repository under OpenTechStrategies. @cwebber, I can make sure you have perms to create new repositories in there if you don't have them already.

[cwebber]

@kfogel It's just "the directory in which one is doing all this" but we can look at changing that up later if need be.

I'm glad you're interested in breaking it out into its own FOSS repo once we're done with it! I think it makes sense to nail down how it works in TTM first, then once we have a sense of that, break it out.

[kfogel]

Why would we wait until we're done with it to break it out into its own FOSS repo?

http://opentechstrategies.com/resources#be-open-from-day-one

[cwebber]

@kfogel It's mostly because I'm not sure how to "break out" a makefile and what route to take for that when generalizing i. I agree with "be open from day one" generally.

[kfogel]

Hmm. Okay, yeah, I see. If it's easier to map out the problem space by doing it within TTM first, that's fine. Probably the abstraction boundaries will become clearer as you go.