Using docker executor

[tamlyn]

Is it possible to use the docker executor with this image? If so how do I supply the base image and link services?

[pmalek]

+1

[sameersbn]

While I have never tried it, I don't see any reason why it will not work. To use the docker executor, you will need to volume mount your docker unix domain socket into the runner. e.g.

docker run -it --rm \
  <other options>
  -v /var/run/docker.sock:/run/docker.sock \
  sameersbn/gitlab-ci-multi-runner

edit: I am not sure how gitlab-ci-multi-runner talks to docker. If it requires the docker binary to be present then you can add -v $(which docker):/bin/docker to the run command.

see https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/install/docker.md for more information

[hourliert]

Hi,

It seems that the @sameersbn solution doesn't work. It would be awesome if we could use the docker executor.

Otherwise, is it possible to do some 'nested docker'?

[sameersbn]

I have not tried this yet. I will give it a shot and let you know.

BTW, if you are eager to use docker as the executor, you can simply run the gitlab-ci-multi-runner binary on the host. Its a single binary. This image is more of a base image to build your self contained runner images in which case the shell executor makes more sense.

However, I will try the docker executor and let you know my findings.

[sameersbn]

problem was with access to the docker socket. Also during registration gitlab-ci-multi-runner asks some more questions. Dunno if they can be skipped.

First time run using:

docker run --name runner -it --rm \
  -v /run/docker.sock:/run/docker.sock \
  -v /srv/docker/gitlab-ci-multi-runner:/home/gitlab_ci_multi_runner/data \
  sameersbn/gitlab-ci-multi-runner

It will ask you to provide the relevant details.

I was able to trigger builds in docker containers. I have not tested an actual build, but the docker executor works.

FWIW i used the following build description

test:
  script:
  - pwd
  - ls -la

[tamlyn]

Thanks for this. I've pulled the latest image and I'm using the following command to start a runner:

docker run -d --name gitlab-ci-multi-runner --restart=always \
--volume /opt/gitlab-ci-multi-runner:/home/gitlab_ci_multi_runner/data \
--volume /var/run/docker.sock:/var/run/docker.sock \
--env='CI_SERVER_URL=http://192.168.99.100:10080/ci' \
--env='RUNNER_TOKEN=[redacted]' \
--env='RUNNER_DESCRIPTION=Docked runner' \
--env='RUNNER_EXECUTOR=docker' \
--env="DOCKER_IMAGE=node:0.12" sameersbn/gitlab-ci-multi-runner:0.5.5

But when I try to run a build I get:

gitlab-ci-multi-runner 0.5.5-1-g69bc934 (69bc934)
Using Docker executor with image node:0.12 ...
Pulling docker image node:0.12 ...

ERROR: Build failed with: dial unix /var/run/docker.sock: permission denied

I can get it working using the gitlab/gitlab-runner image but that requires executing gitlab-runner register separately and I'd like to have the whole thing automated from docker-compose.yml. Any ideas?

[sameersbn]

oh wait.. i was supposed to do another change. completely skipped my mind.

[sameersbn]

Have made a required change. Please check if it fixes your issue. I was able to test using your docker run command.

[tamlyn]

Hmmm. Now I get groupadd: GID '100' already exists in the container log after startup. I'm using boot2docker 1.8.2 on OSX.

[sameersbn]

oh.. then I may have to add some more checks.

[sameersbn]

have introduce some more changes. Please pull in a couple of minutes (its building) and retry.

[tamlyn]

Awesome! That's working now. Thanks for all your help. 👍

[Analect]

@tamlyn I hope you don't mind me jumping in here. I subscribed to this thread earlier, since I was trying to get a better understanding of how a runner could be set up with docker-compose ... something that it seems you have been able to do. Would you be able to offer a quick summary here of how you were able to get this working ... it seems you are using a base node image to create a runner dynamically ... or am I misunderstanding? I have extended the docker-compose.yml in the sameersbn/docker-gitlab repo to include the runner? Thanks.

[tamlyn]

I haven't done a full end to end run through yet but I'll post back here when I do.

[Analect]

Much appreciated. Thanks.

[sameersbn]

one thing i did notice that the runner image needs to have git installed. But you must have figured that by now :)

[tamlyn]

@Analect try this http://tamlyn.org/2015/10/dockerised-gitlab-ci-runner/

I may well have missed some steps so let me know if you get stuck or something is wrong.

[Analect]

@tamlyn Much appreciated. Will work through this and revert back.

[Analect]

@tamlyn Sorry for long delay in responding to this and thanks again for documenting this. I worked through your blog-post linked above, which was very thorough. The only step that is explicity missing is the need to redeploy the gitlab container too .. since the GITLAB_HOST gets changed to reflect the container end-point on tutum.

The build is failing as per screenshot below. Any idea what I might be missing. Is the fact that the 'runner' container doesn't have docker installed? ... or is it something else.

I notice in the docker-compose ... I presumably need to change the CI_SERVER_URL ... also to gitlab-1.gitlab.xxxxx.cont.tutum.io/ci ... although, even if I do this, the build is still erroring-out as above.

The other thing I'm finding with tutum's script editor ... is that it randomly changes the docker-compose.yml to wrap certain elements in single quotes ... as strings ... but not others ... have you had any issues with that?

Thanks, Colum

gitlab:
  image: 'sameersbn/gitlab:8.0.3'
  environment:
    - GITLAB_HOST=gitlab-1.gitlab.xxxxx.cont.tutum.io
    - GITLAB_PORT=10080
    - GITLAB_SECRETS_DB_KEY_BASE=long-and-random-alphanumeric-string
  links:
    - postgresql
    - 'redis:redisio'
  ports:
    - '10080:80'
  volumes:
    - '/srv/docker/gitlab/gitlab:/home/git/data'
node-runner:
  image: 'sameersbn/gitlab-ci-multi-runner:latest'
  environment:
    - 'CI_SERVER_URL=http://gitlab/ci'
    - 'DOCKER_IMAGE=node:0.12'
    - 'RUNNER_DESCRIPTION=Node runner'
    - RUNNER_EXECUTOR=docker
    - RUNNER_TOKEN=385072abd2d887484011
  volumes:
    - /var/run/docker.sock
    - '/opt/gitlab-ci-multi-runner:/home/gitlab_ci_multi_runner/data'
postgresql:
  image: 'sameersbn/postgresql:9.4-7'
  environment:
    - DB_NAME=gitlabhq_production
    - DB_PASS=password
    - DB_USER=gitlab
  volumes:
    - '/srv/docker/gitlab/postgresql:/var/lib/postgresql'
redis:
  image: 'sameersbn/redis:latest'
  volumes:
    - '/srv/docker/gitlab/redis:/var/lib/redis'

[outcoldman]

@Analect the latest issue can be caused by the not root user, I would guess that adding something like

    - GITLAB_CI_MULTI_RUNNER_USER=root

should fix this problem