Docker Docker Docker - Githubissues

migurski commented 7 years ago

This in-progress PR will address issue #562, and after merge will likely mark version 5.0.0 of OA Machine.

[x] Update Circle and Travis CI configurations to use Docker instead of Chef
[x] Merge @iandees changes from PR #547, and ensure that @jalessio suggestions are being used
[x] Merge @MiniCodeMonkey changes from PR #159, and adapt README changes with local Docker use documentation
[ ] Modify EC2 userdata scripts to use Docker instead of Chef
[x] Modify Docker Hub images to work for testing and deployment

These two recent CI builds for commit 251a1941bf demonstrate working use of Docker:

trescube commented 7 years ago

Hey, sorry, this fell off my radar. I tested this tonight and followed what I gathered are the instructions:

ran docker pull openaddr/prereqs || true
ran docker build -f Dockerfile-prereqs -t openaddr/prereqs:latest .
ran docker build -f Dockerfile-testing -t openaddr/testing:latest .
ran docker-compose up -d && sleep 15 (apparently postgres needs a little time)
ran docker-compose run machine python3 /usr/local/src/openaddr/test.py

Here's the output of the last command:

[stephenhess@slumlord machine (migurski/docker-docker-docker)]$ docker-compose run machine python3 /usr/local/src/openaddr/test.py
.............................................................................................................................................................................................................................................................................
----------------------------------------------------------------------
Ran 269 tests in 70.808s

OK

That all looks great! Now, what do I do with it? Can I submit sources to the local machine or do machine dev work?

migurski commented 7 years ago

Thanks @trescube, that’s perfect! In theory, you should be able to do both. I’m not 100% clear on how to best configure or recommend volume mapping as with the Vagrant setup, but that would be the right thing to set up. Could I talk to you about this sometime?

trescube commented 7 years ago

Absolutely, you're west coast and I'm east coast, so pretty much any time after 1:30pm eastern works for me. It's pretty easy to use on vagrant now but any way to make it even easier is great with me. Just hit me up on slack! I'm off on Friday but available on Thursday (3/13).

migurski commented 7 years ago

Okay — I'm running from thing to thing tomorrow, but I might be able to carve out some time to hang out and chat.

migurski commented 7 years ago

I rewrote the Vagrant portions of the installation docs to reflect possible new Docker behavior:

https://github.com/openaddresses/machine/blob/fa2ea47a7290/docs/install.md

I’d love a review on this from @trescube and @jalessio — I’m a bit stuck on the role of Docker vs. Docker-compose here, and could use some advice on how to think about this. For example, we use Docker-compose to supply PostGIS so that the tests all pass, but it’s not obvious to me how to also use it to support a volume-mapping workflow so that local changes can be run and checked.

migurski commented 7 years ago

Based on advice, I rewrote the local development instructions to use docker-compose instead of plain Docker:

https://github.com/openaddresses/machine/blob/c08a550/docs/install.md

migurski commented 7 years ago

This seems to work well in EC2 instance user data:

#!/bin/sh
apt-get update -y
apt-get install -y docker.io
docker pull openaddr/machine

migurski commented 7 years ago

This is working well for userdata:

#!/bin/sh -ex
apt-get update -y
apt-get install -y docker.io awscli

docker pull openaddr/machine
aws s3 cp s3://data.openaddresses.io/… /tmp/environment
docker run --env-file /tmp/environment openaddr/machine openaddr-ci-worker -b data.openaddresses.io -v &
docker run --env-file /tmp/environment openaddr/machine openaddr-ci-worker -b data.openaddresses.io -v &
wait

Possible changes:

Map /tmp volume to let the larger space be used
Build an AMI with docker.io and awscli installed

migurski commented 7 years ago

I am likely to go live with this as version 5.0.0 of Machine, initially applying it to Worker instances only. I’m planning a crossover period, and then getting rid of the Chef stuff at the end of the crossover at 6.0.0.

nvkelso commented 7 years ago

Congrats!

On Apr 19, 2017, at 18:03, Michal Migurski notifications@github.com wrote:

I am likely to go live with this as version 5.0.0 of Machine, initially applying it to Worker instances only. I’m planning a crossover period, and then getting rid of the Chef stuff at the end of the crossover at 6.0.0.

― You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

openaddresses / machine

Docker Docker Docker #587