gocd / docker-gocd-agent

GoCD Agent Docker image
https://hub.docker.com/r/gocd/
47 stars 37 forks source link

Add the ability to overwrite VOLUME_DIR in the entrypoint #63

Closed aerostitch closed 6 years ago

aerostitch commented 6 years ago

Hi,

Context: We are trying to use the gocd image (alpine3.7) on a host to run several agents at the same time. We do run tasks in our pipeline involving docker-compose. To achieve this we install docker-compose, docker and share the docker unix socket as a volume from the host which is the commonly recommended way.

Problem: When you mount a volume in docker-compose, example a folder named conf, the volume is created by the docker daemon, which resides on the host, not the container, so it is the host that looks for the path (which is transformed by the client in a full path) of the conf file on the host and not the container.

To make it work we mount volumes on the host that have to match the path of the container.

Now the trick is that when you run several containers on the same host you'd like to avoid having several containers writing in the same directories at the same time (especially log files, pipelines directories, config which are the files that are in /godata) as it can have very unexpected consequences especially if you run a job in parallel on several agents.

Proposed solution: So one of the solution would be to be able to set the VOLUME_DIR to a path that is specific to each agent like /var/lib/gocd/agent-1/godata for the 1st one, /var/lib/gocd/agent-2/godata for the 2nd one etc. (We're running those as pods on a standalone kubelet for each instance so we just have static pods with an incremented number at the end of their name, so environment variables can really easily be set)

I'm not sure if it's the recommended solution but that's how we have been running our dev and prod containers over the last year with our custom image and it worked well. We're trying to use the standard image now as much as possible and this change does seem to be the less painful way to do it.

Adding it inside a script on entrypoint.d will require basically to drop almost everything that's been done in the standard docker-entrypoint.sh and redo it, so that doesn't seem to be the best idea.

Let me know what you think about it.

Thanks Joseph

varshavaradarajan commented 6 years ago

@aerostitch - can you please update the README?

aerostitch commented 6 years ago

Sure.

aerostitch commented 6 years ago

@varshavaradarajan I added a paragraph explaining the specifics about running docker-compose and docker inside a gocd-agent container. Let me know if that's clear.

ketan commented 6 years ago

Thanks for the PR @aerostitch. The next version of the image that gets built (18.7) will have your changes in it.

aerostitch commented 6 years ago

Awesome, thanks! :)