Networking problems with 8.7.6 image

[qwc]

Hi,

I've got problems getting the 8.6.6 image to run.

gitlab_1           | Initializing logdir...
gitlab_1           | Initializing datadir...
gitlab_1           | Installing configuration templates...
gitlab_1           | SSL keys and certificates were not found.
gitlab_1           | Assuming that the container is running behind a HTTPS enabled load balancer.
gitlab_1           | Configuring gitlab...
gitlab_1           | Configuring gitlab::database
gitlab_1           | ERROR: 
gitlab_1           |   Please configure the database connection.
gitlab_1           |   Refer http://git.io/wkYhyA for more information.
gitlab_1           |   Cannot continue without a database. Aborting...
gitlab_1           | 
gitlab866_gitlab_1 exited with code 0

gitlab 8.6.5-1 is running after i've updated my docker-compose.yml with - DB_ADAPTER=postgresql

Here's the complete config:

postgresqlgitlab:
  image: sameersbn/postgresql:9.4-19
  environment:
    - DB_USER=<user>
    - DB_PASS=<password>
    - DB_NAME=gitlabhq_production
    - DB_EXTENSION=pg_trgm
  volumes:
    - /srv/data/docker/gitlab/postgres/data:/var/lib/postgresql
  restart: always
gitlab:
  image: sameersbn/gitlab:8.6.5-1
  ports:
    - "10080:80"
    - "42420:22"
  links:
    - redisgitlab:redisio
    - postgresqlgitlab:postgresql
  working_dir: /home/git/gitlab
  environment:
    - GITLAB_SECRETS_DB_KEY_BASE=<secret>
    - GITLAB_HOST=gitlab.<domain>
    - GITLAB_PORT=443
    - GITLAB_HTTPS=true
    - GITLAB_EMAIL=gitlab@<domain>
    - DB_ADAPTER=postgresql
    - DB_NAME=gitlabhq_production
    - DB_TYPE=postgres
    - DB_USER=<user>
    - DB_PASS=<password>
    - GITLAB_SSH_PORT=42420
    - GITLAB_BACKUP_SCHEDULE=weekly
    - GITLAB_BACKUP_EXPIRY=3628800
  volumes:
    - /srv/data/docker/gitlab/data:/home/git/data
  restart: always
redisgitlab:
  image: sameersbn/redis:latest
  restart: always

Is this problem on my side (wrong config) or on the image side? :)

[solidnerd]

Perhaps you forget to set the following variables:

• DB_HOST
• DB_PORT

Which docker version do you use ? Could you provide a docker info ?

[sameersbn]

@qwc I tested your docker-compose.yml file and found no issues. can you try removing the sameersbn/gitlab:8.6.5-1 image and re-pulling? If that does not help then please try adding DEBUG=1 to the env of the gitlab container in the compose file.

[qwc]

sry, copied the 8.6.5-1 config but I am having problems with 8.6.6! (same config just changed 8.6.5-1 to 8.6.6)

DEBUG=1 doesn't give more output than without.

Already repulled it, but also this doesn't change the mentioned nonworking state.

Here the requested docker info output.

$ docker info
Containers: 20
 Running: 2
 Paused: 0
 Stopped: 18
Images: 63
Server Version: 1.10.3
Storage Driver: devicemapper
 Pool Name: docker-254:2-4720671-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 7.962 GB
 Data Space Total: 107.4 GB
 Data Space Available: 99.41 GB
 Metadata Space Used: 11.62 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.136 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.120 (2016-03-11)
Execution Driver: native-0.2
Logging Driver: json-file
Plugins: 
 Volume: local
 Network: bridge null host
Kernel Version: 4.4.5-1-ARCH
Operating System: Arch Linux
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 23.55 GiB
Name: thorium.<domain>
ID: <id>
Username: qwick
Registry: https://index.docker.io/v1/

[sameersbn]

DEBUG=1 doesn't give more output than without.

Sorry. please set DEBUG=true

[qwc]

Whew, yes that's a little bit mooore information. :)

Here you'll find the output: https://gist.github.com/qwc/3511bf650dd66575fc2ca14c42e9645c

[sameersbn]

@qwc the log suggests that the automatic discovery of database parameters from the docker links is not happening and as a result the DB_HOST and DB_PORT variables are not being set as @SolidNerd suggested. Can you please verify your docker compose file to check if the links have been setup, i.e.

  links:
    - redisgitlab:redisio
    - postgresqlgitlab:postgresql

If it has then there may be some issue occurring that I do not understand. You should be able to resolve this issue by setting DB_HOST=postgresql and REDIS_HOST=redisio

[qwc]

There's really something fishy going on. I've checked the containers with docker inspect and on the container from the image gitlab:8.6.6 the network devices are not created.

That's also the problem I'm facing, your suggestions should have solved the issue, when docker would do everything like expected. The docker-compose.yml files are all the same except the changed version to 8.6.6 and your suggested env variables, with 8.6.5-1 everything works, with 8.6.6 not.

I'm quite perplexed with this...

(closing issue, due to error in docker not in gitlab. :) )

(I'll update the system later today, or next week, maybe a fresher version of docker may fix this?)

[qwc]

I'm sorry to reopen this issue, but the problems I'm facing are still present with my today tried upgrade to 8.7.6. (Did not have time to further investigate the 8.6.6 problem and thought it would go away with 8.7.6...)

With DEBUG=true i see following timeout occur:

gitlab_1            | + /usr/lib/postgresql/9.5/bin/pg_isready -h postgresql -p 5432 -U gitlab -d gitlabhq_production -t 1
gitlab_1            | ++ expr 52 - 1
gitlab_1            | + timeout=51
gitlab_1            | + [[ 51 -eq 0 ]]

Is there a slight chance that the current gitlab awaits a 9.5 postgresql database instead of the 9.4-21 of yours? But of course a lot of other users do not have that problems. So it has to do something with my config... When I run docker-compose config no networks are listed and every service has got network_mode: bridge, so maybe here's the problem ? What's your config saying about networks and the networkmode? Could post an example?

[qwc]

Found an issue at docker which looks like my problem here. https://github.com/docker/docker/issues/18113

Will close as soon as I've worked around this issue.

[qwc]

Well I've solved the problem, here's what I've done:

1. First I've updated the configuration to version: '2'
2. On creating the serivce containers with docker-compose create I got following error:
   • ERROR: b'network <some_network_name> not found'
   • So I assumed that the automatic creation of service networks does NOT work.
3. Creating the networks by hand with docker network create -d bridge some_network_name solved this problem easily, but it's a manual step! Of course maybe only once.