search

ailispaw / boot2docker-xhyve

Boot2Docker running on xhyve hypervisor
294 stars 25 forks source link

nameserver 192.168.64.1 on OSX is not working correctly #23

Open bpinto opened 9 years ago

bpinto commented 9 years ago

I don't have access to the internet when I start the machine:

docker@boot2docker:~$ curl google.com
curl: (6) Could not resolve host: google.com

docker@boot2docker:~$ cat /etc/resolv.conf
nameserver 192.168.64.1

docker@boot2docker:~$ echo nameserver 8.8.8.8 > /etc/resolv.conf

docker@boot2docker:~$ curl google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="http://www.google.pt/?gfe_rd=cr&amp;ei=IVE7VrfKH_Gr8weGpKTgDw">here</A>.
</BODY></HTML>

But even with the changes above, this still does not work:

docker@boot2docker:~$ docker search ubuntu
Error response from daemon: Get https://index.docker.io/v1/search?q=ubuntu: dial tcp: lookup index.docker.io on 192.168.64.1:53: read udp 192.168.64.1:53: connection refused
ailispaw commented 9 years ago

Hi @bpinto ,

It's strange that it sill looks up index.docker.io on 192.168.64.1:53 after your modification. Did you restart the VM after the modification of resolve.conf? It's not persistent in boot2docker except Docker storage.

Your original problem is with nameserver 192.168.64.1 which is running on your Mac OSX. Normally it works fine, but it seems something wrong with your host processes. Please provide your OSX version and boot2docker version. And also could you look at /etc/bootpd.plist while the VM is running?

Here is mine;

$ cat /etc/bootpd.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Subnets</key>
    <array>
        <dict>
            <key>_creator</key>
            <string>com.apple.NetworkSharing</string>
            <key>allocate</key>
            <true/>
            <key>dhcp_domain_name_server</key>
            <array>
                <string>192.168.64.1</string>
            </array>
            <key>dhcp_router</key>
            <string>192.168.64.1</string>
            <key>interface</key>
            <string>bridge100</string>
            <key>lease_max</key>
            <integer>86400</integer>
            <key>lease_min</key>
            <integer>86400</integer>
            <key>name</key>
            <string>192.168.64/24</string>
            <key>net_address</key>
            <string>192.168.64.0</string>
            <key>net_mask</key>
            <string>255.255.255.0</string>
            <key>net_range</key>
            <array>
                <string>192.168.64.2</string>
                <string>192.168.64.254</string>
            </array>
        </dict>
    </array>
    <key>bootp_enabled</key>
    <false/>
    <key>detect_other_dhcp_server</key>
    <array>
        <string>bridge100</string>
    </array>
    <key>dhcp_enabled</key>
    <array>
        <string>bridge100</string>
    </array>
    <key>dhcp_ignore_client_identifier</key>
    <true/>
    <key>ignore_allow_deny</key>
    <array>
        <string>bridge100</string>
    </array>
    <key>use_server_config_for_dhcp_options</key>
    <false/>
</dict>
</plist>
bpinto commented 9 years ago

It's strange that it sill looks up index.docker.io on 192.168.64.1:53 after your modification. Did you restart the VM after the modification of resolve.conf? It's not persistent in boot2docker except Docker storage.

I didn't restart.

OSX: 10.11 Beta (15A278b)

Git: * 0940c82 - (HEAD, origin/master, origin/HEAD, master) Upgrade boot2docker to v1.9.0 (2 days ago) <A.I>

Boot2Docker version 1.9.0, build master : 16e4a2a - Tue Nov  3 19:49:22 UTC 2015
Docker version 1.9.0, build 76d6bc9
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Subnets</key>
    <array>
        <dict>
            <key>_creator</key>
            <string>com.apple.NetworkSharing</string>
            <key>allocate</key>
            <true/>
            <key>dhcp_domain_name_server</key>
            <array>
                <string>192.168.64.1</string>
            </array>
            <key>dhcp_router</key>
            <string>192.168.64.1</string>
            <key>interface</key>
            <string>bridge100</string>
            <key>lease_max</key>
            <integer>86400</integer>
            <key>lease_min</key>
            <integer>86400</integer>
            <key>name</key>
            <string>192.168.64/24</string>
            <key>net_address</key>
            <string>192.168.64.0</string>
            <key>net_mask</key>
            <string>255.255.255.0</string>
            <key>net_range</key>
            <array>
                <string>192.168.64.2</string>
                <string>192.168.64.254</string>
            </array>
        </dict>
    </array>
    <key>bootp_enabled</key>
    <false/>
    <key>detect_other_dhcp_server</key>
    <array>
        <string>bridge100</string>
    </array>
    <key>dhcp_enabled</key>
    <array>
        <string>bridge100</string>
    </array>
    <key>dhcp_ignore_client_identifier</key>
    <true/>
    <key>ignore_allow_deny</key>
    <array>
        <string>bridge100</string>
    </array>
    <key>use_server_config_for_dhcp_options</key>
    <false/>
</dict>
</plist>

In case you would like to see the same file before I started the server (it doesn't seem to have changed): http://dpaste.com/14XB24S

ailispaw commented 9 years ago

Thanks for the info. Here is in my case before starting and after shutting down the VM.

$ cat /etc/bootpd.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>bootp_enabled</key>
    <false/>
    <key>detect_other_dhcp_server</key>
    <false/>
    <key>dhcp_enabled</key>
    <false/>
    <key>dhcp_ignore_client_identifier</key>
    <true/>
    <key>use_server_config_for_dhcp_options</key>
    <false/>
</dict>
</plist>

It seems something else which uses NetworkSharing is running in your system. But the file itself looks good, so DHCP server 192.168.64.1 should be fine. You may be able to execute nslookup against 192.168.64.1 on OSX, right? Or not?

And I guess there is something wrong in the VM. Anyway once you could use curl google.com with nameserver 8.8.8.8, the network connection is fine. My biggest concern is why it still access to 192.168.64.1 with docker search ubuntu.

When you execute nslookup index.docker.io on the VM with nameserver 8.8.8.8, what is the output?

ailispaw commented 9 years ago

One more thing, could you try https://github.com/ailispaw/docker-root-xhyve as well? I would like to know if this one is fine or not on your system.

bpinto commented 9 years ago

I tried it with success:

[docker@docker-root ~]$ docker search ubuntu
NAME                           DESCRIPTION                                     STARS     OFFICIAL   AUTOMATED
ubuntu                         Ubuntu is a Debian-based Linux operating s...   2564      [OK]
ubuntu-upstart                 Upstart is an event-based replacement for ...   41        [OK]
torusware/speedus-ubuntu       Always updated official Ubuntu docker imag...   25                   [OK]
...

FYI, this happens on both repositories:

⋊> docker-root-xhyve on master ◦ make up                                                                                                                                  01:42:54
Password:
Booting up...
160:168: syntax error: Expected end of line but found identifier. (-2741)
make: *** [up] Error 1
⋊> docker-root-xhyve on master ◦ sudo ./xhyverun.sh                                                                                                                       01:42:59
"/Users/xxx" -network 192.168.64.0 -mask 255.255.255.0 -alldirs -mapall=501:20
Stopping the nfsd service (use 'disable' to make permanent)
Starting the nfsd service
Starting DockerRoot VM
....
bpinto commented 9 years ago

Sorry, I don't know much about network.

When you execute nslookup index.docker.io on the VM with nameserver 8.8.8.8, what is the output?

docker@boot2docker:~$ nslookup index.docker.io
Server:    192.168.64.1
Address 1: 192.168.64.1

nslookup: can't resolve 'index.docker.io'
docker@boot2docker:~$ echo nameserver 8.8.8.8 > /etc/resolv.conf
docker@boot2docker:~$ nslookup index.docker.io
Server:    8.8.8.8
Address 1: 8.8.8.8 google-public-dns-a.google.com

Name:      index.docker.io
Address 1: 52.5.36.25 ec2-52-5-36-25.compute-1.amazonaws.com
Address 2: 52.1.242.236 ec2-52-1-242-236.compute-1.amazonaws.com
Address 3: 54.173.111.219 ec2-54-173-111-219.compute-1.amazonaws.com

You may be able to execute nslookup against 192.168.64.1 on OSX, right? Or not?

Is this what you meant?

⋊> boot2docker-xhyve on master ◦ nslookup 192.168.64.1                                                                                                                    01:52:36
Server:     8.8.8.8
Address:    8.8.8.8#53

** server can't find 1.64.168.192.in-addr.arpa.: NXDOMAIN
bpinto commented 9 years ago

By the way, I just got internet working on boot2docker:

docker@boot2docker:~$ echo nameserver 8.8.8.8 > /etc/resolv.conf
docker@boot2docker:~$ sudo /etc/init.d/docker restart
docker@boot2docker:~$ docker search ubuntu

Which explains why the nslookup worked but not the docker search. :joy:

I don't know why I need to do it on boot2docker but not on docker-root...

ailispaw commented 9 years ago

Sorry for my poor English.

I mean; On OSX,

$ nslookup
> server 192.168.64.1
Default server: 192.168.64.1
Address: 192.168.64.1#53
> index.docker.io
Server:     192.168.64.1
Address:    192.168.64.1#53

Non-authoritative answer:
index.docker.io canonical name = elb-io.us-east-1.aws.dckr.io.
elb-io.us-east-1.aws.dckr.io    canonical name = us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com.
Name:   us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com
Address: 52.1.242.236
Name:   us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com
Address: 54.173.111.219
Name:   us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com
Address: 52.5.36.25
>
ailispaw commented 9 years ago

docker@boot2docker:~$ sudo /etc/init.d/docker restart

Oh! I see.

ailispaw commented 9 years ago

160:168: syntax error: Expected end of line but found identifier. (-2741) make: *\ [up] Error 1

Hmm.. I will figure it out.

ailispaw commented 9 years ago

@bpinto That syntax error is from AppleScript with osascript in bin/xhyveexec.sh. Which terminal application do you use? Terminal or iTerm?

ailispaw commented 9 years ago

I don't know why I need to do it on boot2docker but not on docker-root...

/etc/resolve.conf in docker-root is as follows by default.

[docker@docker-root ~]$ cat /etc/resolv.conf
# Generated by dhcpcd from eth0.dhcp
# /etc/resolv.conf.head can replace this line
nameserver 192.168.64.1
nameserver 8.8.8.8
nameserver 8.8.4.4
options single-request-reopen
bpinto commented 9 years ago

Which terminal application do you use? Terminal or iTerm?

I'm using iTerm.

/etc/resolve.conf in docker-root is as follows by default.

:+1: That explains it.

On OSX:

⋊> boot2docker-xhyve on master ◦ nslookup                                                                                                                                 13:36:00
> server 192.168.64.1
Default server: 192.168.64.1
Address: 192.168.64.1#53
> index.docker.io
;; connection timed out; no servers could be reached

I didn't expect this! I have just replaced my router... Forgot to add Google's DNS to it! Now I have configured it and nslookup has the expected output...

⋊> boot2docker-xhyve on master ◦ nslookup                                                                                                                                 13:39:30
> index.docker.io
Server:     8.8.8.8
Address:    8.8.8.8#53

Non-authoritative answer:
index.docker.io canonical name = elb-io.us-east-1.aws.dckr.io.
elb-io.us-east-1.aws.dckr.io    canonical name = us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com.
Name:   us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com
Address: 52.5.36.25
Name:   us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com
Address: 52.1.242.236
Name:   us-east-1-elbio-rm5bon1qaeo4-623296237.us-east-1.elb.amazonaws.com
Address: 54.173.111.219

But, same problem inside the VM:

docker@boot2docker:~$ docker search ubuntu
Error response from daemon: Get https://index.docker.io/v1/search?q=ubuntu: dial tcp: lookup index.docker.io on 192.168.64.1:53: read udp 192.168.64.1:53: connection refused
ailispaw commented 9 years ago

The test on OSX was whether the DNS server 192.168.64.1 is working.

⋊> boot2docker-xhyve on master ◦ nslookup                                                                                                                                 13:36:00
> server 192.168.64.1
Default server: 192.168.64.1
Address: 192.168.64.1#53
> index.docker.io
;; connection timed out; no servers could be reached

The result means that the DNS server on your OSX is dead or not running correctly. That's why you can not reach the server from the VM, either.

In conclusion, the VM itself is working fine, but your local DNS server in your host OSX is not somehow. You have to fix the issue in OSX side, not the VM. It may be caused by 10.11 BETA?

However, until the fix, you can use the VM by setting 8.8.8.8 as your DNS server instead of the local, as you have already found. Another issue is that the modification for resolve.conf is not persistent across reboot and you have to restart the docker daemon as you showed me. To solve this, you need to add the following line into the /var/lib/boot2docker/bootsync.sh which runs as a bootstrap before the Docker daemon starts.

echo "nameserver 8.8.8.8" > /etc/resolv.conf

And then reboot the VM.

bpinto commented 9 years ago

Thanks!

girvo commented 8 years ago

Hey @ailispaw I have a similar error

➜  code git:(master) ✗ nslookup
> server 192.168.64.1
Default server: 192.168.64.1
Address: 192.168.64.1#53
> index.docker.io
;; connection timed out; no servers could be reached

That's on my Mac. I wonder why that is?

If I ping 192.168.64.1:

  code git:(master) ✗ ping 192.168.64.1
PING 192.168.64.1 (192.168.64.1): 56 data bytes
64 bytes from 192.168.64.1: icmp_seq=0 ttl=64 time=0.054 ms
64 bytes from 192.168.64.1: icmp_seq=1 ttl=64 time=0.054 ms
64 bytes from 192.168.64.1: icmp_seq=2 ttl=64 time=0.137 ms
^C
--- 192.168.64.1 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.054/0.082/0.137/0.039 ms
ailispaw commented 8 years ago

Hmm. I don't have a time now. I will check it out again later. Thanks.

okket commented 8 years ago

FWIW: I ran in the same problem, but in my case I guess my local unbound DNS resolver listening only on 127.0.0.1 / ::1 was the problem. It was solved once I enabled unbound to listen on 0.0.0/0 and ::0/0 (don't forget to adjust allow also) .

(Related unbound setup gist: https://gist.github.com/denji/82dad8dee994ea197ea3)

ailispaw commented 8 years ago

This workaround may work for it. https://github.com/nlf/dlite/pull/151

ailispaw commented 8 years ago

https://github.com/ailispaw/boot2docker-xhyve/issues/22#issuecomment-175007899

kim0 commented 7 years ago

In my case I was using dnsCrypt and disabling that made minishift work well again!

ailispaw commented 5 years ago

For me, in addition to the above, I needed the following command to get IP address correctly.

$ sudo /bin/launchctl load -w /System/Library/LaunchDaemons/bootps.plist
$ sudo /bin/launchctl unload -w /System/Library/LaunchDaemons/bootps.plist

It will activate bootpd to create /var/db/dhcpd_leases somehow.