drnic / docker-server-boshrelease

Cluster of Docker Servers deployed with BOSH
1 stars 1 forks source link

Cannot access index #2

Open drnic opened 10 years ago

drnic commented 10 years ago
$ export DOCKER_HOST=localhost
$ docker run ubuntu:13.04 echo hello world
Unable to find image 'ubuntu:13.04' (tag: 13.04) locally
Pulling repository ubuntu
2014/02/15 06:53:14 pull: Get https://index.docker.io/v1/repositories/ubuntu/images: dial tcp: lookup index.docker.io on [10.235.3.253]:53: no such host

But normal curl works from the same machine:

$ curl https://index.docker.io/v1/repositories/ubuntu/images
[{"checksum": "tarsum+sha256:8a76588341423e26d1f422208c5b3667a281594a203c0c2bdf71e17ffabd91bf...
drnic commented 10 years ago

The DNS IP address mentioned is the first of two in resolv.conf

$ cat /etc/resolv.conf 
domain compute-1.internal
search compute-1.internal
nameserver 10.235.3.253
nameserver 172.16.0.23

Perhaps docker doesn't move on to the 2nd nameserver?

@crosbymichael should docker use all nameservers in /etc/resolv.conf? Or do I have to explicitly setup docker -d -dns to duplicate the nameservers?

drnic commented 10 years ago

@crosbymichael for your reference, the first DNS is an internal BOSH DNS; the 2nd one should be an AWS public DNS.

$ dig @172.16.0.23 +short index.docker.io
edge-dockerio.dotcloud.com.
54.234.135.251
54.224.119.89
drnic commented 10 years ago

Hmm I explicitly added the two nameservers to docker with -dns but still not finding the docker index:

$ ps ax
...
 5942 ?        S<l    0:00 docker -d -p /var/vcap/sys/run/docker_server/docker_server.pid -g /var/vcap/data/docker -H tcp://127.0.0.1:4243 -H unix:///var/vcap/sys/run/docker_server/docker.sock -dns 10.235.3.253 -dns 172.16.0.23 --restart=true --mtu=0

$ docker run ubuntu:13.04 echo hello world
Unable to find image 'ubuntu:13.04' (tag: 13.04) locally
Pulling repository ubuntu
2014/02/15 07:13:42 pull: Get https://index.docker.io/v1/repositories/ubuntu/images: dial tcp: lookup index.docker.io on [10.235.3.253]:53: no such host
drnic commented 10 years ago

Ok, I don't think I know what's going on. I removed -dns 10.235.3.253 to try to force docker to use the public DNS -dns 172.16.0.23 but it is still using the private DNS and erroring.

drnic commented 10 years ago

@hatofmonkeys do you have public DNS working in your https://github.com/hatofmonkeys/decker-release work?

crosbymichael commented 10 years ago

@drnic let me check

drnic commented 10 years ago

To confirm, if I remove the first private nameserver from /etc/resolv.conf and restart docker -d then it works normally and can resolve public hostnames from the public DNS.

drnic commented 10 years ago

@crosbymichael how does docker resolve DNS? A standard golang library or something specifically written for docker?

crosbymichael commented 10 years ago

@drnic docker does not resolve dns, if your host system has a resolv.conf it will use that, if not than it will inject 8.8.8.8. We don't resolve anything.

drnic commented 10 years ago

Ok. Hmm, ideas why it's stopping at the first nameserver and not trying the second one?

/CC @frodenas

On Tue, Feb 18, 2014 at 9:43 AM, Michael Crosby notifications@github.com wrote:

@drnic docker does not resolve dns, if your host system has a resolv.conf it will use that, if not than it will inject 8.8.8.8. We don't resolve anything.

Reply to this email directly or view it on GitHub: https://github.com/drnic/docker-server-boshrelease/issues/2#issuecomment-35411412

crosbymichael commented 10 years ago

Off the top of my head I thought it will take the full resolv.conf but you or I will have to look in the code to make sure.

drnic commented 10 years ago

I'll use it as an excuse for a code reading exercise. See if I can figure out if there's anything in docker/go that might be related.

On Tue, Feb 18, 2014 at 9:45 AM, Michael Crosby notifications@github.com wrote:

Off the top of my head I thought it will take the full resolv.conf but you or I will have to look in the code to make sure.

Reply to this email directly or view it on GitHub: https://github.com/drnic/docker-server-boshrelease/issues/2#issuecomment-35411643

crosbymichael commented 10 years ago

The container's resolv.conf should live in /var/lib/docker/containers//resolv.conf if you want to see what it looks like

frodenas commented 10 years ago

@drnic At your PowerDns instance, add an external recursor at /var/vcap/jobs/powerdns/config/pdns.conf:

recursor=172.16.0.23