Start groups of containers as an atomic unit

[jhspaybar]

I have interest in contributing a patch to deimos that would allow for an add-on style similar to Heroku when local dependencies are needed.

As an example of what I had in mind, you create a rails container, but want nginx starting in front of it. Rather than install nginx inside the rails container, they should be 2 separate containers, linked together. Then, if a local memcached were desired, it could be linked into the rails container. I've given some thought to how the definition should work and would be interested in discussing it further if there's interest.

The most basic example would be something like the following

container: { image: "docker:///nginx" options: ["--graph", "[{ \"key\": \"myrailsapp\", \"image\": \"docker:///myrailsapp\", \"options\": [], \"links\": [\"memcached\"], \"volumes-from\": []},{ \"key\": \"memcached\", \"image\": \"docker:///memcached\", \"options\": [], \"links\": [], \"volumes-from\": []}]"]

Essentially a list of nodes with an adjacency list for links and volumes. The cluster of containers would start or fail as a single atomic unit. Let me know what you guys think!

[wkf]

How will Deimos support links in general? Even more fundamental than starting groups of containers atomically, can you use 'links' when containers are different machines? Also, how would this play with service registration and discovery when using something like Marathon? Would Deimos expose port mappings?

[jhspaybar]

No, links are a local machine only type of thing. A 'link' to a memcached on another box will actually be a link to an ambassador that knows where the memcached actually is, or for other dependencies, a link to a haproxy, or etcd, for example. To me, there are some things that should be started on the local machine, one is an nginx proxy, another might be a logging util(think fluentd in file tail mode). You want it in it's own process space since fluentd going haywire shouldn't bring down your rails app, but it also needs to share a volume with your rails app and therefor be local.

As far as marathon, and port mappings, in the example given above, the only externally exposed port would be one from nginx, it will then have a shared volume with rails(for static files, or a socket), and/or a shared port with the rails application. The only externally discoverable port for this application would be nginx's, though the application would be "myrailsapp" when you registered it with Marathon most likely.

[tarnfeld]

Have you considered running docker inside docker? There's no reason (imo) that you shouldn't do this, it's now fully supported (with privileged containers) and would allow you to acheive this. The image that you launch the task container with simply needs to have docker installed, and within the wrapping container you launch two more containers, one running nginx and the other running rails, and link the two together.

[jhspaybar]

I hadn't thought about containers in containers, mostly because I don't want to give the containers I'm starting privileged permissions since it seems it'd allow containers to affect other containers and access the host machine. I suppose it might be possible to generate the boot script in the privileged container based on user configuration and have this boot container be the thing that actually starts. This gets around some of the security problems and gets me the behavior I want, and stays completely out of deimos.

[tarnfeld]

mostly because I don't want to give the containers I'm starting privileged permissions since it seems it'd allow containers to affect other containers and access the host machine

Although this is true in some ways, privileges containers only have extra access to do certain things with the host. I believe there are some discussions happening within the docker community regarding the -priv option to separate out some things (like creating dev devices for fuse mounts, as an example) making it a little less boolean.

Alternatively you could use the --lxc-conf command line option and give special privileges that are only the ones needed for running a docker container, though i'm not too knowledgable on this specifically.

May I ask why you can't just run the two services in the same container? You can always write some upstart scripts and launch upstart as your PID 1 process (the one docker invokes) to launch both nginx and rails, avoiding the need to do this entirely.

[jhspaybar]

I want to avoid running two processes in the same container because they're not always all required to run. For example, consider a rails application container that decides to also start a log application scanner/puller in the same container. A misbehaving log scanner can take down my website, if they're in two containers, I can limit the log scanner to some amount of CPU and the worst that might happen is it kills itself, or doesn't have enough CPU to keep up with scanning.

[solidsnack]

This seems like it would be best handled by the scheduler. Deimos does provide support for passing links, via the options field of the ContainerInfo. Note that these options are, at present, passed as-is to Docker.

[rasputnik]

Yeah this sounds more like a scheduler level thing than an executors responsibility. Not sure marathon has that concept of groups fully baked yet, Aurora is a bit further on with that path.

[solidsnack]

Closing as not relevant to Deimos.