Support launching Linux containers (LXC)

[hedgehog]

Albeit Linux specific, it would be usefult to be able to launch VM's that are LXC.

libvirt seems to have some support, so not sure what the Vagrant burden would look like in brining this to fruition.

[eterps]

That would be a great feature!

[mitchellh]

Hedge,

I would love to do this, but like I've said with everything else related to other hypervisors: This will have to wait until after 1.0. :) (even though lxc isn't a hypervisor, of sorts)

Best, Mitchell

[mitchellh]

As I said in the last comment: After 1.0. I'm trying to clean up any open issues to be specific to <= 1.0. Closing.

[alexlist]

Hi Mitchell,

any chance this can be reopened after VMware support is done? ;)

It would be cool to be virtualization agnostic ...

[eterps]

I agree, it would be cool to be virtualization agnostic. In fact, at the moment I stopped using vagrant because lxc is so much faster than VirtualBox, it saves me a huge amount of time.

[mitchellh]

This is most definitely planned very soon after VMWare Fusion. :) The requests for this are high.

[sutyrin]

I second this :)

[bearnard]

+1

[ayosec]

+1

[goblin]

+2

[albertohm]

+1

[Judit]

+1

[groteck]

+1

[thomasw]

+1

[jslang]

:+1:

[danpaulson]

:heartpulse:

[frankscholten]

+1

[DavidKang]

+1

[kishiamy]

+1

[Nightcrawler09]

+1

[nabariho]

+1

[PandyTheBroh]

+1 and if it´s posible +1000000000000000000000000000000000000000

[mvbehr]

+1

[hh]

+1

[glenux]

+1

[lorello]

+1

[someara]

+2

[werdan]

+1

[iflowfor8hours]

+1

[sandello]

+1

[rrichards]

+1

On Dec 18, 2012, at 1:44 AM, Ivan Pouzyrevsky notifications@github.com wrote:

+1

— Reply to this email directly or view it on GitHub.

[daneroo]

+1, what's a good way to help out ?

[LeonB]

+1

[taqtiqa-mark]

@mitchellh, given the high level of interest in a closed issue: Is there a reason this is still closed and not open with a feature tag? @daneroo asks a good question: How can we help?

[mitchellh]

@taqtiqa-mark You raise a good point. I plan on keeping this issue closed because I think soon after I release and document the provider interface that is nearly complete for an upcoming 1.1 release, the community will very quickly write an LXC provider.

All I can say right now is please be patient for just a LITTLE BIT longer. 1.1 is around the corner (basically just needs documentation and polish) and then creating new backend providers should be clear.

[LeonB]

Awesome! Sounds good to me.

[taqtiqa-mark]

Thanks @mitchellh. Shout out if there is a todo list you can do with some help on to get 1.1 out the door (there don't appear to be any issues tagged 1.1).

[benolee]

I don't know if this is helpful, but cucumber-chef has a nice script for setting up lxc https://github.com/Atalanta/cucumber-chef/blob/master/chef_repo/cookbooks/cucumber-chef/recipes/lxc.rb

[taqtiqa-mark]

Thanks for the heads-up @benolee. If I have read it correctly, that should help in developing a recipe that allows pre-1.1 guest-VM's to run LXC.

For others interested in this, here are some efforts, in no particular order:

• https://github.com/niklas/chef-lxc/network
• https://github.com/hw-cookbooks/lxc/network
• https://github.com/pwelch/chef-lxc

[zpatten]

:+1:

[zpatten]

I'm willing to contributing to an LXC effort for Vagrant. I've acquired some extensive experience with it while working on Cucumber-Chef.

[taqtiqa-mark]

@zpatten, from your expereince, which of the cookbooks above would is the best? Or start from scratch and pull only parts from each?

[zpatten]

@taqtiqa-mark, depends on what you want. The one I wrote in Cucumber-Chef would likely need some tweaking to be used standalone. Mine gets LXC up and running, configures the bridge interface (has lots of issues letting LXC do this bit) and a few other items (including dhcpd+bind for the containers). It doesn't set up "static" containers thou; the actual creation, destruction, starting, stopping, etc of the LXC's is done by other parts of Cucumber-Chef.

So It really depends on what you want; any of the others might work better out of the box and might definitely be better suited, but if you wanted to roll your own I imagine you might want to draw some insight from all of them.

[hh]

I have another approach in https://github.com/ii-cookbooks. It's tied together at a higher level at https://github.com/hh/cloud-kitchen

It started out as a class-room environment and it's at a bit higher level, as It actually starts by creating a bootable USB (http://www.instantinfrastructure.org/2013/01/instant-infrastructure-on-usb.html / http://www.youtube.com/watch?v=FaukaaVwm2c)

https://github.com/ii-cookbooks/ii-usb/blob/master/recipes/_format_and_mount_usb.rb#L21-27

    parted -s ${USB} mklabel msdos 
    parted -s -- ${USB} mkpart primary fat32 2 #{node['ii-usb']['partition-size']}
    # the rest of the USB is for persistent
    parted -s -- ${USB} mkpart primary ext3 #{node['ii-usb']['partition-size']} -1
    parted -s -- ${USB} set 1 boot on
    mkfs.vfat -n '#{node['ii-usb']['volume-name']}' ${USB}1

Booting the resultant USB on most hardware does a full install using the Ubuntu Ubiquity installer and a seed file and ensures the filesystem is btrfs:

https://github.com/ii-cookbooks/ii-usb/blob/master/templates/default/preseed.seed.erb#L107

Networking is setup with a bridge with dnsmasq for dhcp and dns. The lxc bridge is also connected to a vbox2lxc interface, so you can bring up windows boxes in Virtualbox and they use the same dhcp/dns as your lxc.

https://github.com/ii-cookbooks/ii-lxc/blob/master/templates/default/lxc-net.conf.erb#L43-46

ip tuntap add dev vbox2lxc mode tap
brctl addif ${LXC_BRIDGE} vbox2lxc
ifconfig vbox2lxc up
dnsmasq -u lxc-dnsmasq --domain=<%=node['virtualization']['domain']%>,${LXC_NETWORK},local --strict-order --bind-interfaces --pid-file=${varrun}/dnsmasq.pid --conf-file= --listen-address ${LXC_ADDR} --dhcp-range ${LXC_DHCP_RANGE} --dhcp-lease-max=${LXC_DHCP_MAX} --dhcp-no-override --except-interface=lo --bridge-interface=${LXC_BRIDGE}:0,${LXC_BRIDGE} --interface=${LXC_BRIDGE} --domain-needed --log-queries || cleanup

A special /usr/local/sbin/dnsmasq that is picked up by network-manager, so that any network interface you share to others (wireless or wired) can reach the lxc network and dnsmasq for resolution.

https://github.com/ii-cookbooks/ii-lxc/blob/master/templates/default/dnsmasq.erb

#!/bin/sh
# Change '--config-file' to '-C /usr/local/etc/dnsmasq.conf'
if (echo $* | grep nm-dns-dnsmasq); then
   exec /usr/sbin/dnsmasq `echo $* | sed 's:$: --server=//<%=node['virtualization']['hostip']%> --server=/<%=node['virtualization']['domain']%>/<%=node['virtualization']['hostip']%> --local=/<%=node['virtualization']['network'].split('.').reverse.join('.')%>.in-addr.arpa/<%=node['virtualization']['hostip']%> --local=/local/127.0.0.1#5353 --log-queries:'`
fi
exec /usr/sbin/dnsmasq `echo $*`

I wanted to create model-containers for easy cloning, because if lxc can detect btrfs it is pretty much instantanious. But cloning is done via the lxc-clone script which basically snapshots the btrfs, munged /etc/hosts and /etc/hostname. so I had to do some stuff like:

https://github.com/ii-cookbooks/ii-lxc/blob/master/templates/default/lxc-training.erb#L729-734

cat <<EOF > $rootfs/etc/rc.local # do this on every boot
current_hostname=\$(hostname)
cat<<EOS > /etc/hosts
# /usr/bin/lxc-clone sets the host file all wacky on a clone...
127.0.0.1   localhost
127.0.1.1   \${hostname}.<%= node['resolver']['search'] %> \$hostname

and my templates have a parameter to install chef and to install ssh keys (pub and private).

These lxc-templates are instantiated as models (model-target, model-workstation):

https://github.com/ii-cookbooks/ii-chef-server/blob/master/recipes/classroom-models.rb

The workstation get's created as as chef-client, while the target has no chef. (students bootstrap it)

I then wait for students to sign up for class and based on the student-name and their org-name I clone the model-workstation into chris-workstation, and the model-target to opscode-target1, opscode-target2, etc. I usually just share out an ethernet interface and ask my students to connect via dhcp. Their hardware machines can reach theirownname-worksation, theirorg-targetX etc etc.

I can bring up a 45 clones, 15 of them running vnc/X desktop, all dynamically named, and they can ssh to each pingable by dns name in about 15 seconds. I've actually brought up 250 clones, though you have to start tweaking the max kernel sysctl settings here and there.

I suggesting btrfs and creating models for Vagrant, as it is similar to the .box file import. The networking approach here allowing all hosts (including natted vbox and natted real ethernet/wifi) to resolve and reach the Vagrant upped local cloud is nice.

[hh]

I tried running bind and even djbdns, but dnsmasq is simple and does what we need here. I've also thought about using this btrfs approach to cloning Virtalbox VMs in place rather than reimporting. If not that, then maybe using a shallow Virtualbox clone.

[taqtiqa-mark]

@zpatten, thanks. I think a standalone lxc cookbook would be most useful. Would that fit within the scope of Cucumber-Chef or is the use case there preclude a standalone cookbook.

@hh, very interesting. How deeply in chef-server embedded? I'd imagine a standalone lxc cook book would need to support chef-solo use cases.

It seems that there is enough prior art to consider a lxc lwrp.

[hh]

@taqtiqa-mark with the ii-cookbooks chef-server isn't that embedded and all the lxc provisioning is done with chef-solo. The interesting bit is that it brings up chef-server as an lxc. So you could create a model-chef-server and clone it whenever you needed one.

[fgrehm]

Hey all, I've started spiking out on this one, feel free to join me :-)

https://groups.google.com/forum/?fromgroups=#!topic/vagrant-up/fp3UfclJDg8

[fgrehm]

Just a quick update, I've made some major progress and the code is available at https://github.com/fgrehm/vagrant-lxc, there's a lot of things to be done but I've been using it every day since monday last week :-)

[LeonB]

Awesome. Going to try it this weekend with my simple Vagrant box.