Docker builder RFC

[mwhooker]

This document is to gather input from the community both on how packer can most effectively benefit from docker, and how packer can provide the best utility to docker users.

Use Cases

Build docker containers. I want to use the provisioner facilities in packer to generate a docker container as the end result.

Replacement for chroot builders. Chroots, as used by the amazon-chroot builder, are ineffectual as a means of namespacing because processes are still scheduled by the host OS. If sysvinit/upstart/etc start any processes as part of provisioning, the chroot won't be unmountable while the processes are still running. I've attempted a few remedies, including managing policyrc.d, but it's become apparent that this path is too complex to work in the general case.

Proposed implementation

I am still spinning up on docker, so please point out any technical errors in the proposal.

Docker Builder

We can start with a docker builder as the base use case.

Docker uses a build file called a Dockerfile. I believe we can create a docker Communicator which will write to the Dockerfile, though I suspect there's an IPC protocol that's usable.

The final artifact will be a filesystem tarball ie from running docker --export.

Provisioning

There should be a way to seed provisioning with a base docker image.

AMI intermediate.

The chroot replacement use case looks a bit simpler once we have the docker tarball. I have to do more research into packer and docker both, but it seems like the steps should be

1. builder docker image
2. convert exported container to AMI using similar code in the amazon/chroot builder

I don't know if it would be better to do this step as a post-processor or as a separate builder

Comments

I would greatly appreciate feedback, documentation, links to extant code, etc.

[mitchellh]

There was a fairly lengthy discussion in #24 I assume you've read but if you haven't, you should because we've crossed a lot of these bridges already.

Responding to your points, I think there are a lot more Docker integration points:

• Builder: Docker and Dockerfiles - Use a Dockerfile to build a resulting Docker container.
• Builder: Docker without Dockerfiles - Use Docker simply as a container to run provisioners.
• Provisioner: Docker containers from the registry - Pre-pull some docker containers from the registry.

In addition to this I realize you want to create AMIs from Docker containers. I actually have no real knowledge here but I imagine by doing the above that process can become a lot more clear.

[mwhooker]

thanks @mitchellh! I admittedly only glanced over the issue. I'll give it some closer attention but I think we're on the same page. If my intentions weren't clear I'll reword it so they are. I want to do everything you mention wrt the Builder using a Communicator and spitting out a container, etc.

The provisioner is also an excellent point. I want to achieve the same thing in my use-case, but with EBS snapshots. I'll add some points about this.

[fgrehm]

I'm not too familiar with AMIs so here's my 2 cents from the lxc side of things after only 9 months of "lxc experience" :)

My understanding based is that the "source" of both Docker / vagrant-lxc / "plain old lxc" containers are basically a rootfs tarball + some configurations and are usually build out of a debootstrap process. To give you a better idea, here's some scripts used for building the those images:

• "Plain old lxc"
  • https://github.com/lxc/lxc/blob/master/templates/lxc-ubuntu.in
  • it doesn't output a tarball but it builds a rootfs that can be exported into one afterwards
• Docker
  • https://github.com/dotcloud/docker/blob/master/contrib/mkimage-debootstrap.sh
  • uses debootstrap and imports the rootfs back into docker
• vagrant-lxc
  • https://github.com/fgrehm/vagrant-lxc/blob/master/boxes/build-ubuntu-box.sh
  • builds on top of the stock lxc templates that comes with Ubuntu and creates a vagrant image out of a rootfs tarball

I haven't used Packer enough to know if what I have in mind makes sense but the way I see it working is to have a "lxc rootfs tarball builder" (with a better name of course :P) that creates the container and brings it up with lxc-start so that Packer provisioners can kick in to do their work. After provisioners are run it just lxc-stops it and a tarball of the rootfs gets built.

With that tarball around, the user can easily import it into docker with sudo tar --numeric-owner -c . | docker import ... by hand or we can have a vagrant post processor that does whatever is needed to build a vagrant-lxc box. For those using plain old lxc, I think they'll probably be able to find their way of feeding that tarball back into their environment :)

Does that make sense?

EDIT: I've just came across this message on Docker's mailing list and it seems that I'm not alone:

Hmmmm I'm interested in using Packer from the other end; to make a tarball of an OS that can be imported into Docker as well as being exported to EC2, Virtualbox, and other formats. I think I might need to make a new Packer output plugin for that though.

[mitchellh]

@mwhooker Can you describe to me what the issues with the chroot builder are again? I remember issues with process management. Is it not possible to place the entire chroot somehow under a cgroup and control it using that?

[mitchellh]

@mwhooker Also, I agree, I think we're on the same page! :)

[mitchellh]

@fgrehm Thanks!

[mwhooker]

@mitchellh That's essentially the issue. We're looking to cgroups to solve it, but via docker, which has the added benefit of describing a container format so we can use the produced images in different environments.

[mitchellh]

@mwhooker I'm going to work on the Docker builder that doesn't use Dockerfiles.

[mwhooker]

@mitchellh excellent. I'll be in irc if you want some help.

[mitchellh]

@mwhooker I'm working in this branch: https://github.com/mitchellh/packer/tree/f-docker-builder

I just one hour trying to get the Communicator Start method to work, but Docker is not cooperating. In a terminal if you do echo 'echo foo' | docker attach 1234 then it will execute that command and return output. I can't mimic that behavior with os/exec for some reason. I've tried everything I can think of here.

At this point I'm going to call it a roadblock, will sleep on it.

[mwhooker]

nice job. I'll poke at it

[mitchellh]

Updated: I've worked around every issue so far. We're on our way! Progress screenshot shown below:

[mitchellh]

I probably should've made a pull request (sorry), but I merged in the docker builder! /cc @mwhooker

https://github.com/mitchellh/packer/commit/8cc09bcd56560db989661d6c33fe731a5e886bd3

Works as you would expect, is fully documented in the website, etc.

[mitchellh]

I'm actually going to close this since it is in. We can make another for the chroot builder or provisioner when the time comes.