Merge digitallumberjack/master modifications

[digitalLumberjack]

Changes :

• Runner upgraded to 1.10.4
• Docker support (socket mode and dind mode)
• Docker private registry support :
  • Allows to use private images for jobs
• Added RUNNER_DOCKER_ADDITIONAL_VOLUME support to share volumes between host and build containers
• Added RUNNER_OUTPUT_LIMIT support to override the logs output limit.

[sameersbn]

Can you resolve the conflicts please?

[digitalLumberjack]

Should be ok. Sorry for the bad commit message i tried the online editor and i could not choose the message. I can amend if you need.

[sameersbn]

@digitalLumberjack Lets resolve the issues in the PR before we hit the merge button. p.s. i've added you as a collaborator to the project.

[sameersbn]

also i will give it a test before we merge this one. thanks for the contrib

[inongogo]

This is fantastic! I am eagerly looking forward to testing this! Thanks for all your work!

[paolomainardi]

Please merge it, this is a fantastic work!

[digitalLumberjack]

@sameersbn This should be ok now. I bumped to 1.10.4 by the way.

[davidwindell]

LGTM? :+1:

[fwoelffel]

Can this be merged?

[digitalLumberjack]

I bumped to runner v1.11.2, @sameersbn as soon as you give your go we can merge this. Tell me if any modification is needed.

[cpoetter]

@digitalLumberjack @sameersbn Really awesome work you two! My Synology can't wait for this update :) Please press the merge button :D

[BirgerK]

Merging this update would be awesome! :)

[davidwindell]

@digitalLumberjack could the entrypoint detect a change to the RUNNER_DOCKER_IMAGE and update the config?

[davidwindell]

For those waiting for this to be merged, we've just switched all our builds over to digitallumberjack/docker-gitlab-ci-multi-runner:v9.3.0 and it's working grand. Great to finally be on the docker executor! :) Thanks @digitalLumberjack :+1:

[Maescool]

I see @sameersbn has made @digitalLumberjack Collaborator, so you could merge this?

[davidwindell]

@digitalLumberjack could you bump to v9.5?

[QuickJack]

According to https://about.gitlab.com/2017/08/22/gitlab-9-5-released/ the old runner will stop working at September 22nd, 2017.

Please note, that this also affects the usability of docker-gitlab.

[GarbageYard]

@digitalLumberjack: I am getting following error while using the image you've shared:

./configure_proxy_artifactory.sh: line 13: docker: command not found

GitLab CI runner script:

docker run --name gitlab-ci-multi-runner -d --restart=always \
  --volume /var/run/docker.sock:/var/run/docker.sock \
  --volume /mnt/data/gitlab/gitlab-runner:/home/gitlab_ci_multi_runner/data \
  --env='CI_SERVER_URL=https://gitlab.dev.abc.net/ci' --env='RUNNER_TOKEN=RijUZPnMjGeNF2JYt' \
  --env='RUNNER_DESCRIPTION=runnerA' --env='RUNNER_EXECUTOR=docker' \
  --env='RUNNER_DOCKER_IMAGE=docker:17.07.0-ce' --env='RUNNER_DOCKER_MODE=socket' \
  --link gitlab.dev.abc.net \
  digitallumberjack/docker-gitlab-ci-multi-runner:v9.3.0-1

.gitlab-ci.yml content:

image: node:6.11

types:
        - build
        - deploy

build_app:
        type: build
        script: "bash chmod +x ./configure_proxy_artifactory.sh"
        script: "bash ./configure_proxy_artifactory.sh"

From within the container, here's my observation:

root@5489eb3ebe42:/home/gitlab_ci_multi_runner# ls -l /var/run/docker.sock
srw-rw---- 1 root docker 0 Jul 21 00:53 /var/run/docker.sock
root@5489eb3ebe42:/home/gitlab_ci_multi_runner# docker ps
bash: docker: command not found

Any idea how i can fix this?

[Maescool]

@GarbageYard You are using the wrong image for this kind of setup you need at least the following in your .gitlab-ci.yml

image: docker:latest
services:
  - docker:dind

in the Dockerfile you use to build the image, you should refer to the node image.

[GarbageYard]

Thanks for the reply @Maescool! Sorry, but i am confused now because if i'm going to use image: docker:latest, then how will i get the node app? Will i be using some package manager inside the .gitlab-ci.yml file to download the node installer?

[Maescool]

@GarbageYard so, there are a couple of parts. if you want to build a docker image using gitlab-ci-multi-runner, in my case I use docker in docker, so that's why I have the services thingie, you might not need it. you have your .gitlab-ci.yml something like this

image: docker:latest

services:
  - docker:dind

variables:
  GIT_SUBMODULE_STRATEGY: recursive
  CONTAINER_TEST_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME
  CONTAINER_RELEASE_IMAGE: $CI_REGISTRY_IMAGE:latest

before_script:
  - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY

stages:
  - build
  - release

build:
  type: build
  script:
    - docker build --pull -t $CONTAINER_TEST_IMAGE .
    - docker push $CONTAINER_TEST_IMAGE

release-image:
  stage: release
  script:
    - docker pull $CONTAINER_TEST_IMAGE
    - docker tag $CONTAINER_TEST_IMAGE $CONTAINER_RELEASE_IMAGE
    - docker push $CONTAINER_RELEASE_IMAGE
  only:
    - master

(this is basically by default gitlab-ci build file to build docker images)

Then you have your Dockerfile, that looks something like this

FROM node:6.11
COPY . /usr/src/app

The entrypoint stuff etc.. idk with node, and for your project, this is just an example of how I do my image builds.

[GarbageYard]

Hi @Maescool! I tried using DIND but somehow it's failing to retrieve the image via Artifactory (our private container registry).

Runner log error:
Running with gitlab-ci-multi-runner 9.5.0 (413da38)
  on RunnerA (d8ed43a6)
Using Docker executor with image docker.artifactory.abc.net/docker:17.07 ...
Starting service docker.artifactory.abc.net/docker:17.07-dind ...
Pulling docker image docker.artifactory.abc.net/docker:17.07-dind ...
ERROR: Preparation failed: Error response from daemon: Get https://docker.artifactory.abc.net/v2/: x509: certificate signed by unknown authority

I guess it's because it's unable to login to Artifactory first. I saw this link but it seems it's for images and not for services. I still tried configuring Secret Variable (DOCKER_AUTH_CONFIG) but it's not working.

Content of .gitlab-ci.yml

image: docker.artifactory.abc.net/docker:17.07

variables:
  DOCKER_HOST: tcp://docker:2375

# This before_script block was added later but it seems this block
# isn't executed before the DIND tries fetching image from Artifactory
before_script:
  - docker login -u svc-art-user -p some-pwd docker.artifactory.abc.net
  - docker info

services:
- docker.artifactory.abc.net/docker:17.07-dind

build:
  stage: build
  script:
  - docker build -t my-docker-node-image .

All (GitLab, Runner & DIND) containers are on one host. I am able to docker login docker.artifactory.abc.net -u svc-art-user -p some-pwd successfully. Even without DOCKER_AUTH_CONFIG, my image (shown above i.e., docker.artifactory.abc.net/docker:17.07) was getting fetched without any issue. It seems it's because it was using my host's ~/.docker/config.json. If that's the case, why isn't the service (docker.artifactory.abc.net/docker:17.07-dind) also using the same credentials?