search

sameersbn / docker-gitlab-ci-multi-runner

MIT License
145 stars 85 forks source link

How can I interconnect this with gitlab in docker on the same host? #4

Closed pmalek closed 8 years ago

pmalek commented 9 years ago

I have ran you gitlab in docker and I would like to connect multi-runner to it but I am getting

Starting dockergitlabcimultirunner_GitlabCIMultiRunner_1...
Attaching to dockergitlabcimultirunner_GitlabCIMultiRunner_1
GitlabCIMultiRunner_1 | time="2015-09-25T18:41:11Z" level=error msg="Couldn't execute POST against 172.17.42.1/api/v1/runners/register.json: Post 172.17.42.1/api/v1/runners/register.json: unsupported protocol scheme \"\"" 
GitlabCIMultiRunner_1 | time="2015-09-25T18:41:11Z" level=error msg="79e66ea0 Registering runner... failed" 
GitlabCIMultiRunner_1 | time="2015-09-25T18:41:11Z" level=fatal msg="Failed to register this runner. Perhaps you are having network problems" 
.....

I have added external_link in the docker-compose.yml

GitlabCIMultiRunner:
  image: sameersbn/gitlab-ci-multi-runner:0.5.5
  external_links:
    - dockergitlab_gitlab_1
  volumes:
    - /opt/gitlab-ci-multi-runner:/home/gitlab_ci_multi_runner/data
  environment:
    - CI_SERVER_URL=172.17.42.1
    - RUNNER_TOKEN=79e66ea069a011eae67c
    - RUNNER_DESCRIPTION=first_runner
    - RUNNER_EXECUTOR=shell
  restart: always

Am I doing this in correct way? The IP address I have put there is the ifconfig docker0 from gitlab instance. I have tried with:

with the same results

Gitlab containers are running as follows:

CONTAINER ID        IMAGE                        COMMAND                CREATED             STATUS              PORTS                                                                  NAMES
37d346a08f67        sameersbn/gitlab:8.0.2       "/sbin/entrypoint.sh   About an hour ago   Up About an hour    0.0.0.0:10022->22/tcp, 0.0.0.0:10080->80/tcp, 0.0.0.0:32788->443/tcp   dockergitlab_gitlab_1       
e2f3dba05391        sameersbn/redis:latest       "/sbin/entrypoint.sh   21 hours ago        Up About an hour    0.0.0.0:32787->6379/tcp                                                dockergitlab_redis_1        
afd6764750d7        sameersbn/postgresql:9.4-3   "/start"               21 hours ago        Up About an hour    0.0.0.0:32786->5432/tcp                                                dockergitlab_postgresql_1
tamlyn commented 9 years ago

You want to use the full URL including scheme, port and path. So in your case probably http://172.17.42.1:10800. (Are you sure it's not 10080?) Note if you're using Gitlab CE 8 which has Gitlab CI integrated, you need to add /ci/ to the end of the URL so http://172.17.42.1:10800/ci/

pmalek commented 9 years ago

OK I have managed to get it to work by running the following command to get the IP address of gitlab containter:

$ sudo docker exec -it dockergitlab_gitlab_1 ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
196: eth0@if197: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:ac:11:00:32 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.50/16 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:acff:fe11:32/64 scope link 
       valid_lft forever preferred_lft forever

and then I have used the following setting:

CI_SERVER_URL=http://172.17.0.50/ci

with this it works :) Although I wonder why is my ifconfig docker0 IP address different than the one returned from docker exec -it dockergitlab_gitlab_1 ip addr ?

sameersbn commented 9 years ago

Of your docker compose has the config:

ports:
    - "10080:80"

or if your docker run command has -p 10080:80, then you are better off using http://172.17.41.1:10080/ci as the url in your runner configurations. This is because when your host or container is restarted, it will not have the same ip address that it currently has. As a result the runner will no longer be able to connect to Gitlab.

pmalek commented 9 years ago

Ok so I needed to add http:// in the beginning of the URL and add /ci postfix and it works :)

@sameersbn Thanks for pointing this out :)

pmalek commented 9 years ago

Weird things started to happen... I am trying to start multi-runner now and I get this:

sudo docker-compose up
Starting dockergitlabcimultirunner_GitlabCIMultiRunner_1...
Attaching to dockergitlabcimultirunner_GitlabCIMultiRunner_1
GitlabCIMultiRunner_1 | time="2015-09-26T12:27:37Z" level=info msg="0 Starting multi-runner from /home/gitlab_ci_multi_runner/data/config.toml ..." 
GitlabCIMultiRunner_1 | time="2015-09-26T12:27:40Z" level=error msg="Couldn't execute POST against http://172.17.0.50/ci/api/v1/builds/register.json: Post http://172.17.0.50/ci/api/v1/builds/register.json: dial tcp 172.17.0.50:80: no route to host" 
GitlabCIMultiRunner_1 | time="2015-09-26T12:27:40Z" level=warning msg="3863bfa5 Checking for builds... failed"

I have no idea where 172.17.0.50 is taken from since I have the following docker-compose.yml (which was working few hours ago...):

cat docker-compose.yml
GitlabCIMultiRunner:
  image: sameersbn/gitlab-ci-multi-runner:0.5.5
  external_links:
    - dockergitlab_gitlab_1
    - dockergitlab_redis_1
    - dockergitlab_postgresql_1
  volumes:
    - /opt/gitlab-ci-multi-runner:/home/gitlab_ci_multi_runner/data
  environment:
    - CI_SERVER_URL=http://172.17.42.1:10080/ci
    - RUNNER_TOKEN=bda1e3b735dbfb0cae50
    - RUNNER_DESCRIPTION=first_runner2
    - RUNNER_EXECUTOR=shell
  restart: always

NOTE: Ok so this was caused by config files stored in /opt/gitlab-ci-multi-runner (which I have removed to let docker recreate it) on host and in /home/gitlab_ci_multi_runner/data in the container but I am still having issues now as in the beginning namely: 

ifconfig docker0
docker0   Link encap:Ethernet  HWaddr 26:44:89:9d:50:29  
          inet addr:172.17.42.1  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::1821:c2ff:fee3:beeb/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2101 errors:0 dropped:0 overruns:0 frame:0
          TX packets:688 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:160795 (160.7 KB)  TX bytes:122343 (122.3 KB)

------------------------------------------------------------
$sudo docker ps
CONTAINER ID        IMAGE                        COMMAND                CREATED             STATUS              PORTS                                                                  NAMES
37d346a08f67        sameersbn/gitlab:8.0.2       "/sbin/entrypoint.sh   22 hours ago        Up 3 hours          0.0.0.0:10022->22/tcp, 0.0.0.0:10080->80/tcp, 0.0.0.0:32770->443/tcp   dockergitlab_gitlab_1       
e2f3dba05391        sameersbn/redis:latest       "/sbin/entrypoint.sh   42 hours ago        Up 3 hours          0.0.0.0:32769->6379/tcp                                                dockergitlab_redis_1        
afd6764750d7        sameersbn/postgresql:9.4-3   "/start"               42 hours ago        Up 3 hours          0.0.0.0:32768->5432/tcp                                                dockergitlab_postgresql_1   
------------------------------------------------------------
$ sudo docker-compose up
Starting dockergitlabcimultirunner_GitlabCIMultiRunner_1...
Attaching to dockergitlabcimultirunner_GitlabCIMultiRunner_1
GitlabCIMultiRunner_1 | Generating SSH deploy keys...
GitlabCIMultiRunner_1 | Generating public/private rsa key pair.
GitlabCIMultiRunner_1 | Your identification has been saved in /home/gitlab_ci_multi_runner/data/.ssh/id_rsa.
GitlabCIMultiRunner_1 | Your public key has been saved in /home/gitlab_ci_multi_runner/data/.ssh/id_rsa.pub.
GitlabCIMultiRunner_1 | The key fingerprint is:
GitlabCIMultiRunner_1 | b3:51:80:eb:b9:97:5a:45:e3:51:1a:92:f4:25:08:87 gitlab_ci_multi_runner@81e440d893e1
GitlabCIMultiRunner_1 | The key's randomart image is:
GitlabCIMultiRunner_1 | +--[ RSA 2048]----+
GitlabCIMultiRunner_1 | |      .==oo o    |
GitlabCIMultiRunner_1 | |      E.o+ *     |
GitlabCIMultiRunner_1 | |       .  B      |
GitlabCIMultiRunner_1 | |      .  + o     |
GitlabCIMultiRunner_1 | |     . .S o      |
GitlabCIMultiRunner_1 | |      o  =       |
GitlabCIMultiRunner_1 | |       .o.       |
GitlabCIMultiRunner_1 | |      ..o        |
GitlabCIMultiRunner_1 | |      .o         |
GitlabCIMultiRunner_1 | +-----------------+
GitlabCIMultiRunner_1 | 
GitlabCIMultiRunner_1 | Your SSH deploy key is: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvzM0MDOjZ+AaRsJdKgOevK7Qooc3mylQMxBb7DJjWlnzgeRtgkAkRHQAO3z6OG78i9oRE7OVe5DkE9z2J/zKKSy911hsAgYZIE44ybaC7tfwd5Mz/tZM7y37WSlnNmOAg0kmXY4ElRsYWB7gqHwm+XzgVnLqv6sxNH0/x4fvP8SDFIMA6wisT/vjH3BIEx71zSqu1LEj4Tors1AQea2TMlVtrO06POXJYKHHT/P7yoYs3btCo46OKsVw3eODMY/IA25fspXAz5/8d2ZhS3je3mprmxph+7TXiVVJW1eeaSUDzIXSmbkM4P1pqhixnjqPBVGM5jhw8YQ1/djjPepJd gitlab_ci_multi_runner@81e440d893e1
GitlabCIMultiRunner_1 | 
GitlabCIMultiRunner_1 | time="2015-09-26T15:11:31Z" level=error msg="Couldn't execute POST against http://172.17.42.1:10080/ci/api/v1/runners/register.json: Post http://172.17.42.1:10080/ci/api/v1/runners/register.json: dial tcp 172.17.42.1:10080: i/o timeout" 
GitlabCIMultiRunner_1 | time="2015-09-26T15:11:31Z" level=error msg="6926db90 Registering runner... failed" 
GitlabCIMultiRunner_1 | time="2015-09-26T15:11:31Z" level=fatal msg="Failed to register this runner. Perhaps you are having network problems" 
GitlabCIMultiRunner_1 | time="2015-09-26T15:35:41Z" level=error msg="Couldn't execute POST against http://172.17.42.1:10080/ci/api/v1/runners/register.json: Post http://172.17.42.1:10080/ci/api/v1/runners/register.json: dial tcp 172.17.42.1:10080: i/o timeout" 
GitlabCIMultiRunner_1 | time="2015-09-26T15:35:41Z" level=error msg="6926db90 Registering runner... failed" 
GitlabCIMultiRunner_1 | time="2015-09-26T15:35:41Z" level=fatal msg="Failed to register this runner. Perhaps you are having network problems" 
------------------------------------------------------------
cat docker-compose.yml
GitlabCIMultiRunner:
  image: sameersbn/gitlab-ci-multi-runner:0.5.5
  external_links:
    - dockergitlab_gitlab_1
    - dockergitlab_redis_1
    - dockergitlab_postgresql_1
  environment:
    - CI_SERVER_URL=http://172.17.42.1:10080/ci
    - RUNNER_TOKEN=6926db9052d298831000
    - RUNNER_DESCRIPTION=first_runner2
    - RUNNER_EXECUTOR=shell
  restart: always
------------------
sameersbn commented 9 years ago

Couple of things.

  1. the CI_SERVER_URL, RUNNER_TOKEN, RUNNER_DESCRIPTION and RUNNER_EXECUTOR parameters are only applicable at the first run of the container. So if you mount a volume at /home/gitlab_ci_multi_runner/data as recommended, then changes in these parameters will have no effect on the configuration after the first run. This means that if you want to change some configuration, you will need to edit the config.toml file and restart the runner.
  2. The RUNNER_TOKEN made available by GilLab is only valid for a single registration. Once a runner registers with a token, GitLab generates a new token for use with the next runner. Due to this behaviour the image does make any changes to the config.toml after the first run.

In short you need to manually edit the config.toml file and update the url.

sameersbn commented 8 years ago

can we close this?