nginx-proxy / docker-gen

Generate files from docker container meta-data
MIT License
4.45k stars 604 forks source link

dockergen can't reconnect after dockerd restart with live-restore #224

Closed jeanpralo closed 7 years ago

jeanpralo commented 7 years ago

If you have live-restore enabled (https://docs.docker.com/engine/admin/live-restore/) and restart the dockerd daemon, once it is back online dockergen can not connect to the endpoint anymore (at least if you run it inside a container):

Nov 09 22:38:01 ch-210to220 dockerd[28683]: dockergen.1 | 2016/11/09 22:38:01 Watching docker events
Nov 09 22:38:01 ch-210to220 dockerd[28683]: dockergen.1 | 2016/11/09 22:38:01 Error retrieving docker server info: cannot connect to Docker endpoint
Nov 09 22:38:01 ch-210to220 dockerd[28683]: dockergen.1 | 2016/11/09 22:38:01 Error listing containers: cannot connect to Docker endpoint
Nov 09 22:38:02 ch-210to220 dockerd[28683]: dockergen.1 | 2016/11/09 22:38:02 Docker daemon connection interrupted

In order to fix it, I have to restart the container. Note that I am using the nginx-proxy container.

How to reproduce (on Ubuntu Xenial):

1- Enable live-restore: cat > /etc/docker/daemon.json { "live-restore": true }

2- kill -HUP `cat /var/run/docker.pid`

3- service docker restart

4- docker logs -f nginx-proxy

Docker version:

# docker info
Containers: 6
 Running: 6
 Paused: 0
 Stopped: 0
Images: 7
Server Version: 1.12.1
Storage Driver: overlay
 Backing Filesystem: extfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null overlay bridge host
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: apparmor seccomp
Kernel Version: 4.4.0-45-generic
Operating System: Ubuntu 16.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 986.4 MiB
Name: ch-210to220
ID: YOQF:LZPF:WXVV:CJWH:PDKF:6YHQ:IHRJ:DEIA:YGU3:P52R:GR75:7UPP
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Insecure Registries:
 127.0.0.0/8

dockergen version:

# docker-gen --version
0.7.3
jeanpralo commented 7 years ago

Well I guess this is more of a docker issue https://github.com/docker/docker/issues/22789.