search

docker / for-linux

Docker Engine for Linux
https://docs.docker.com/engine/installation/
756 stars 85 forks source link

Cannot get external DNS working for container inside of Swarm #789

Closed mcblum closed 5 years ago

mcblum commented 5 years ago

Expected behavior

That the Swam DNS server used in the overlay network would forward requests on.

Actual behavior

Even though the host can resolve domain names correctly, the container cannot.

Output of docker version:

Client: Docker Engine - Community
 Version:           19.03.2
 API version:       1.40
 Go version:        go1.12.8
 Git commit:        6a30dfc
 Built:             Thu Aug 29 05:28:19 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.2
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.8
  Git commit:       6a30dfc
  Built:            Thu Aug 29 05:26:54 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Output of docker info:

Client:
 Debug Mode: false

Server:
 Containers: 14
  Running: 9
  Paused: 0
  Stopped: 5
 Images: 17
 Server Version: 19.03.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: active
  NodeID: xx82uu4ckvn5a3bts5jp8httt
  Is Manager: true
  ClusterID: kbtvp6dltryidno7z8cg1bizr
  Managers: 1
  Nodes: 3
  Default Address Pool: 10.0.0.0/8
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 6
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: 192.168.50.75
  Manager Addresses:
   192.168.50.75:2377
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
 init version: fec3683
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 4.4.0-164-generic
 Operating System: Ubuntu 16.04.6 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 3.841GiB
 Name: docker-swarm01
 ID: Z2MZ:L6P2:VUDE:G2MK:3RLZ:VTU4:EZGO:LZEM:AS7S:3CNJ:E4LY:EDSG
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No swap limit support

Additional environment details (AWS, VirtualBox, physical, etc.) We're running our own swarm for a dev environment, and after a few days of working on this I can't figure out how to get it to work. One of our Node services requires a connection to MongoAtlas, and I cannot resolve the DNS entry for it. Looking at the logs, it appears that every once in a while it does actually resolve, which is very weird.

Running nslookup google.com inside the container gives me

root@4ee1f97fef59:/app# nslookup google.com
;; connection timed out; no servers could be reached

resolv.conf:

nameserver 127.0.0.11
options ndots:0

If anyone has any idea what's happening here, I've been at this since Sunday and just cannot figure it out. I've also tried multiple different base images, installing different things during the build, pretty much anything I can think of.

Thank you!

arkodg commented 5 years ago

@mcblum are you specifying any extra DNS arguments Do you have a search domain in your /etc/resolv.conf, if yes then the ndots:0 will not append the name and search in that domain which might cause your internal DNS names to fail

mcblum commented 5 years ago

@arkodg the nodes needed a restart for some reason. I know... so simple.