R&D: Containerized deployments with docker-machine and AWS

[hancush]

If we do not identify a container orchestration service that feels right to us in #19, an intermediate step could be to leverage docker-machine to provision hosts and administer containerized applications.

Docker Machine is a tool that lets you install Docker Engine on virtual hosts, and manage the hosts with docker-machine commands. You can use Machine to create Docker hosts on your local Mac or Windows box, on your company network, in your data center, or on cloud providers like Azure, AWS, or DigitalOcean.

...

Typically, you install Docker Machine on your local system. Docker Machine has its own command line client docker-machine and the Docker Engine client, docker. You can use Machine to install Docker Engine on one or more virtual systems. These virtual systems can be local (as when you use Machine to install and run Docker Engine in VirtualBox on Mac or Windows) or remote (as when you use Machine to provision Dockerized hosts on cloud providers). The Dockerized hosts themselves can be thought of, and are sometimes referred to as, managed “machines”.

See also: Amazon Web Services (AWS) EC2 example

One immediate challenge is that docker-machine was created as a quick and easy way for a single developer to manage machines, i.e., there's not a native way to support access by multiple developers. There are work arounds, but the first question I'd like to answer is: Do we want to move from one work around to another?

If the answer is Yes, invest some R&D time to stand up an application using this pattern.

[jeancochrane]

Really excited for this! One immediate thought I have is that it may be more useful to us to explore Docker Machine as a utility for provisioning instances, as opposed to managing running containers. (That is, we might end up only using Docker Machine once or twice per project to provision the EC2 instances that run our containerized environments.) This may help resolve the technical hurdle that prevents multiple devs from using Docker Machine to configure remote hosts, since only one dev needs to provision instances in a project anyway.

For container management, we can always shell into the instance to run commands against containers, and a big lesson from https://github.com/datamade/devops/issues/90 is that we want to move away from using shell/CLI access to manage/monitor our containers and toward GUIs like CloudWatch wherever possible. To me the most exciting promise of Docker Machine is the idea of quickly provisioning a Docker-enabled EC2 instance that we can stand up our application in, and doing it with a scripted CLI.

All of that being said, I think you have a lot more context here than I do, so I look forward to being proven wrong if it looks like Docker Machine will be useful for managing running containers too!

[hancush]

holding on this until heroku pilot is complete. at that time, will close or explore this as fallback.

[jeancochrane]

Heroku pilot is complete and adoption is recommended, so we're going to close this until it seems interesting again.