adding manager to docker-ce 17.09 fails on AWS ubuntu

[Gustavo-Maurizio]

Description

Creating a swarm cluster works fine in 17.06 but does not in 17.09/10 or 11. Can create swarm manager and add workers, but not add another manager or promote worker to manager. In all cases the same error shows: ERRO[2017-12-25T11:21:18.873626672Z] cluster exited with error: manager stopped: can't initialize raft node: rpc error: code = Internal desc = connection error: desc = "transport: x509: certificate is valid for 10.203.16.82, not swarm-manager" DEBU[2017-12-25T11:21:18.873965442Z] FIXME: Got an API for which error does not match any expected type!!!: rpc error: code = Internal desc = connection error: desc = "transport: x509: certificate is valid for 10.203.16.82, not swarm-manager" error_type="status.statusError" module=api ERRO[2017-12-25T11:21:18.873986002Z] Handler for POST /v1.32/swarm/join returned error: manager stopped: can't initialize raft node: rpc error: code = Internal desc = connection error: desc = "transport: x509: certificate is valid for 10.203.16.82, not swarm-manager" DEBU[2017-12-25T11:21:18.874034324Z] FIXME: Got an API for which error does not match any expected type!!!: rpc error: code = Internal desc = connection error: desc = "transport: x509: certificate is valid for 10.203.16.82, not swarm-manager" error_type="status.statusError" module=api Steps to reproduce the issue:

1. install apt-get install docker-ce=17.09.1~ce-0~ubuntu
2. create manager 1 - docker swarm init
3. add worker, works fine with docker swarm join --token worker-token manager1ip:2377
4. either adding manager with manager-token, or promoting the worker from step 3 fail.

Describe the results you received: root@ip-203-16-21:~# docker swarm join --token SWMTKN-1-2mihdqwmde3sr242c1w6vize6h9gduyv0prjrznizke0weh37t-ayuwrk82u0qgdblp95hc8uway 10.203.16.82:2377 Error response from daemon: manager stopped: can't initialize raft node: rpc error: code = Internal desc = connection error: desc = "transport: x509: certificate is valid for 10.203.16.82, not swarm-manager"

Describe the results you expected: success in adding manager

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

root@ip-203-16-21:~# docker version
Client:
 Version:      17.09.1-ce
 API version:  1.32
 Go version:   go1.8.3
 Git commit:   19e2cf6
 Built:        Thu Dec  7 22:24:23 2017
 OS/Arch:      linux/amd64

Output of docker info:

root@ip-203-16-21:~# docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 0
Server Version: 17.09.1-ce
Storage Driver: overlay2
 Backing Filesystem: xfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 06b9cb35161009dcb7123345749fef02f7cea8e0
runc version: 3f2f8b84a77f73d38244dd690525642a72156c64
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-1041-aws
Operating System: Ubuntu 16.04.3 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 7.508GiB
Name: ip-203-16-21.dqa.com
ID: V4RT:WIOT:NVUP:SWUO:NF3Y:4DVG:RLGO:DETH:BCCE:BL5A:QRYQ:5EHT
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Http Proxy: http://aws-proxy-qa.cloud.com:8099
Https Proxy: http://aws-proxy-qa.cloud.com:8099
No Proxy: .dqa.capitalone.com,169.254.169.254,.s3.amazonaws.com,.cache.amazonaws.com,,127.0.0.1,localhost,/var/run/docker.sock
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
 10.0.0.0/8
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support

Additional environment details (AWS, VirtualBox, physical, etc.): AWS ubuntu instances in private VPC. works fine with 17.06

[mclamb]

Any updates on this? I am experiencing the same issue with 18.03.0-ce

[mclamb]

The no_proxy fix proposed in https://github.com/moby/moby/issues/35870 works for me, but it doesn't seem like the best solution...

[nishanttotla]

@Gustavo-Maurizio please open this issue on https://www.github.com/docker/swarmkit. Sorry for the trouble.