search

adlogix / docker-machine-nfs

Activates NFS on docker-machine
MIT License
794 stars 104 forks source link

Verify NFS mount fails #22

Closed huggsboson closed 8 years ago

huggsboson commented 8 years ago

In the newest version (as of today, though ) using the 1.9.0a packages I'm getting a verifying NFS mount error on first run through. Though it works on the second invocation.

Here's the output of docker-machine-nfs default: [INFO] machine presence ... OK [INFO] machine running ... OK [INFO] Lookup mandatory properties ... OK

- Machine IP: 192.168.99.100
- Network ID: vboxnet1
- NFSHost IP: 192.168.99.1

[INFO] Configure NFS ...

!!! Sudo will be necessary for editing /etc/exports !!! Password: OK [INFO] Configure Docker Machine ... OK [INFO] Restart Docker Machine ... OK [INFO] Verify NFS mount ... FAIL

Cannot detect the NFS mount :(

dimasch commented 8 years ago

Excellent! Fixed in last pull request https://github.com/adlogix/docker-machine-nfs/pull/19.

1448789032791

huggsboson commented 8 years ago

Hi @gdemad. I believe I was on the latest pull request. Can you try tearing down your docker-machine, starting a new one and try running again? In my original report I stated:

I'm getting a verifying NFS mount error on first run through. Though it works on the second invocation.

Which based on your message "NFS already mounted". Looks like you are on the second run through.

dimasch commented 8 years ago

Hi @box-jhuffaker. Sorry, yes, looks like on the second run. This issue is reproduce in first run.

Thx.

huggsboson commented 8 years ago

@gdemad No problem, thanks for confirming!

huggsboson commented 8 years ago

Few things, I spent the morning debugging this. I somehow got to a point where it eventually wouldn't ever mount the drive on one of my VMs (I cleared /etc/exports and re created and things came together). But generally if I keep restarting the VM it will eventually mount it. (I verify by running the df command like the script). I'm not sure if there's a race between nfsd and restarts or some other issue.

huggsboson commented 8 years ago

I tried moving the restart into the verify loop, but it didn't fix my problem. Which debunks my theory about restarts.

huggsboson commented 8 years ago

I should note: I'm on OSX El Capitan

huggsboson commented 8 years ago

I tried moving the nfsd restart into the verify loop. Still not working.

huggsboson commented 8 years ago

I tried screwing around. Adding an nfsd update... moving boot local to /opt/bootlocal.sh (which seems to be the preferred location on boot2docker 1.9.1 which docker-machine 0.5.1a seems go come with. Also explicitly cleard ~/.docker/machine/cache to remove the boot2docker.iso and caused a re-download.

Basically I'm shooting blindly at things. I'm happy to try things for people that can actually aim fixes, as my machine is reproing.

huggsboson commented 8 years ago

I notice this in /var/log/bootlocal.log: Starting nfs client utilities. mount: RPC: Remote system error - Connection timed out mount: mounting 192.168.99.1:/Users on /Users failed: Bad file descriptor

I'm not sure what state my nfsd was when I started the VM. But this verifies that the bootlocal.sh is in the right place.

dimasch commented 8 years ago

I try remove docker machine and create new instance. See the same result, after start docker-machine-nfs: 1448822486543

Tested with last release updates. Thx @box-jhuffaker :white_check_mark:

dimasch commented 8 years ago

Possible workaround:

I try open new terminal session and see message:

Error running connection boilerplate: Error checking and/or regenerating the certs: There was an error validating certificates for host "192.168.99.100:2376": dial tcp 192.168.99.100:2376: i/o timeout
You can attempt to regenerate them using 'docker-machine regenerate-certs name'.
Be advised that this will trigger a Docker daemon restart which will stop running containers.
  1. docker-machine regenerate-certs dev
  2. docker-machine-nfs dev

Nice mounted! :crystal_ball: d632e508f7

huggsboson commented 8 years ago

I assume this is just because it's the second run. If you remove the regenerate certs step and do the process. I bet it will also work. On Nov 29, 2015 12:02 PM, "Dmitry Schegolihin" notifications@github.com wrote:

Possible workaround:

I try open new terminal session and see message:

Error running connection boilerplate: Error checking and/or regenerating the certs: There was an error validating certificates for host "192.168.99.100:2376": dial tcp 192.168.99.100:2376: i/o timeout You can attempt to regenerate them using 'docker-machine regenerate-certs name'. Be advised that this will trigger a Docker daemon restart which will stop running containers.

  1. docker-machine regenerate-certs dev
  2. docker-machine-nfs dev

Nice mounted! [image: :crystal_ball:] [image: d632e508f7] https://cloud.githubusercontent.com/assets/414067/11459302/9114085e-96e4-11e5-99c9-21effc6e61a5.jpg

— Reply to this email directly or view it on GitHub https://github.com/adlogix/docker-machine-nfs/issues/22#issuecomment-160450349 .

tonivdv commented 8 years ago

Hey guys,

I upgraded to latest docker engine, with latest docker machine, latest virtualbox on OS X El Capitan and it all works without issues ...

$ dm create --driver virtualbox tst
Running pre-create checks...
Creating machine...
(tst) OUT | Creating VirtualBox VM...
(tst) OUT | Creating SSH key...
(tst) OUT | Starting VirtualBox VM...
(tst) OUT | Starting VM...
Waiting for machine to be running, this may take a few minutes...
Machine is running, waiting for SSH to be available...
Detecting operating system of created instance...
Detecting the provisioner...
Provisioning created instance...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
To see how to connect Docker to this machine, run: docker-machine env tst
$ docker-machine-nfs tst
[INFO] machine presence ...             OK 
[INFO] machine running ...          OK 
[INFO] Lookup mandatory properties ...      OK 

    - Machine IP: 192.168.99.101 
    - Network ID: vboxnet4 
    - NFSHost IP: 192.168.99.1 

[INFO] Configure NFS ... 

 !!! Sudo will be necessary for editing /etc/exports !!! 
Password:
                        OK 
[INFO] Configure Docker Machine ...         OK 
[INFO] Restart Docker Machine ...       OK 
[INFO] Verify NFS mount ...             OK 

--------------------------------------------

 The docker-machine 'tst'
 is now mounted with NFS!

 ENJOY high speed mounts :D

--------------------------------------------

Are you sure the docker-machine-nfs script is up to date on your machine?

dimasch commented 8 years ago

Really, after upgrade docker engine, see nice work! 


➜  ~  docker -v
Docker version 1.9.1, build a34a1d5
➜  ~  docker-machine -v
docker-machine version 0.5.1 (7e8e38e)
tonivdv commented 8 years ago

@gdemad and it still doesn't work?

dimasch commented 8 years ago

@tonivdv no, It works fine! :+1:

Can be closed this issue.

Thx,

tonivdv commented 8 years ago

Okay great :)

huggsboson commented 8 years ago

Broken for me...

Macbook pro, retina. VirtualBox 5.0.10 r104061. I installed the extensions.

jhuffaker@mbp-003138 ~ $ docker -v Docker version 1.9.1, build a34a1d5 jhuffaker@mbp-003138 ~ $ docker-machine -v docker-machine version 0.5.1 (7e8e38e) jhuffaker@mbp-003138 ~ $ curl https://raw.githubusercontent.com/adlogix/docker-machine-nfs/master/docker-machine-nfs.sh | sudo tee /usr/local/bin/docker-machine-nfs > /dev/null && sudo chmod +x /usr/local/bin/docker-machine-nfs % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 9229 100 9229 0 0 11397 0 --:--:-- --:--:-- --:--:-- 11393 jhuffaker@mbp-003138 ~ $ docker-machine create --driver=virtualbox tst && /usr/local/bin/docker-machine-nfs tst Running pre-create checks... Creating machine... (tst) OUT | Creating VirtualBox VM... (tst) OUT | Creating SSH key... (tst) OUT | Starting VirtualBox VM... (tst) OUT | Starting VM... Waiting for machine to be running, this may take a few minutes... Machine is running, waiting for SSH to be available... Detecting operating system of created instance... Detecting the provisioner... Provisioning created instance... Copying certs to the local machine directory... Copying certs to the remote machine... Setting Docker configuration on the remote daemon... To see how to connect Docker to this machine, run: docker-machine env tst [INFO] Configuration:

- Machine Name: tst
- Shared Folder: /Users
- Force: false

[INFO] machine presence ... OK [INFO] machine running ... OK [INFO] Lookup mandatory properties ... OK

- Machine IP: 192.168.99.114
- Network ID: vboxnet1
- NFSHost IP: 192.168.99.1

[INFO] Configure NFS ...

!!! Sudo will be necessary for editing /etc/exports !!! OK [INFO] Configure Docker Machine ... OK [INFO] Restart Docker Machine ... OK [INFO] Verify NFS mount ... FAIL

Cannot detect the NFS mount :(

huggsboson commented 8 years ago

Here's /etc/exports: jhuffaker@mbp-003138 ~ $ cat /etc/exports

"/Users" 192.168.99.112 -alldirs -mapall=563785388:1221986466 "/Users" 192.168.99.113 -alldirs -mapall=563785388:1221986466 /Users 192.168.99.114 -alldirs -mapall=563785388:1221986466

huggsboson commented 8 years ago

Here's the tst bootlocal log: jhuffaker@mbp-003138 ~ $ docker-machine ssh tst "cat /var/log/bootlocal.log" Starting nfs client utilities.

huggsboson commented 8 years ago

I ran this manually from the command line: jhuffaker@mbp-003138 ~ $ docker-machine ssh tst "sudo mount -t nfs -o noacl,async 192.168.99.1:/Users /Users"

Just seems to hang.

huggsboson commented 8 years ago

boot2docker info: Boot2Docker version 1.9.1, build master : cef800b - Fri Nov 20 19:33:59 UTC 2015 Docker version 1.9.1, build a34a1d5

huggsboson commented 8 years ago

I'm starting to think my VM just can't reach my host laptop for some reason.

huggsboson commented 8 years ago

It just takes a while. Both bootlocal.log and my manually run mount command ended with: jhuffaker@mbp-003138 ~ $ docker-machine ssh tst "cat /var/log/bootlocal.log" Starting nfs client utilities. mount: RPC: Remote system error - Connection timed out mount: mounting 192.168.99.1:/Users on /Users failed: Bad file descriptor

tonivdv commented 8 years ago

@box-jhuffaker Did you already reboot your mac? When nfsd was hanging this somehow unblocked the situation and never happened again.

huggsboson commented 8 years ago

Oh I really haven't! I'll throw in a reboot and get back to you.

jhuffaker@mbp-003138 ~ $ uptime 8:12 up 13 days, 21:05, 5 users, load averages: 1.75 1.91 1.96

On Mon, Nov 30, 2015 at 7:11 AM, Toni Van de Voorde < notifications@github.com> wrote:

@box-jhuffaker https://github.com/box-jhuffaker Did you already reboot your mac? When nfsd was hanging this somehow unblocked the situation and never happened again.

— Reply to this email directly or view it on GitHub https://github.com/adlogix/docker-machine-nfs/issues/22#issuecomment-160656738 .

huggsboson commented 8 years ago

A restart fixed it, thanks for the tip and all of your work on this script. Glad I got a pretty good root cause in the end.

tonivdv commented 8 years ago

@box-jhuffaker glad it works. Cheers

nhooey commented 8 years ago

Rebooting fixed the issue for me as well, @huggsboson.

huggsboson commented 8 years ago

Glad to hear that, @nhooey.

ghost commented 8 years ago

Reboot not worked for me =/

$ docker-machine create --driver virtualbox dev
...
$ docker-machine-nfs dev
[INFO] Configuration:

    - Machine Name: dev
    - Shared Folder: /Users
    - Force: false

[INFO] machine presence ...             OK
[INFO] machine running ...          OK
[INFO] Lookup mandatory properties ... OK

    - Machine IP: 192.168.99.100
    - Network ID: vboxnet0
    - NFSHost IP: 192.168.99.1

[INFO] Configure NFS ...

 !!! Sudo will be necessary for editing /etc/exports !!!
Password:
The nfsd service does not appear to be running.
Starting the nfsd service
                        OK
[INFO] Configure Docker Machine ...         OK
[INFO] Restart Docker Machine ...       OK
[INFO] Verify NFS mount ...             FAIL

Cannot detect the NFS mount :(

The docker-machine-nfs script stucks here [INFO] Restart Docker Machine ..., so I check docker-machine ls:

$ docker-machine ls
NAME   ACTIVE   URL          STATE     URL   SWARM   DOCKER    ERRORS
dev    *        virtualbox   Running                 Unknown   No IP address found 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 08:00:27:4d:fe:0f brd ff:ff:ff:ff:ff:ff
    inet6 fe80::a00:27ff:fe4d:fe0f/64 scope link tentative
       valid_lft forever preferred_lft forever

Additional info:

$ cat /etc/exports
/Users 192.168.99.100 -alldirs -mapall=501:20

$ docker-machine --version
docker-machine version 0.5.6, build

$ docker --version
Docker version 1.9.1, build a34a1d5
ghost commented 8 years ago

Second (!) run of docker-machine-nfs with patch from issue #31 (sleep 20 after create bootlocal.sh) seems to work for me.

docker@dev:~$ cat /var/lib/boot2docker/bootlocal.sh
#!/bin/sh
  sudo umount /Users
    sudo mkdir -p /Users
  sudo /usr/local/etc/init.d/nfs-client start
    sudo mount -t nfs -o noacl,async 192.168.99.1:/Users /Users
tonivdv commented 8 years ago

@nenemoianemoi Ok thanks for sharing. I guess I will have to put those 20s until docker-machine answers my question about a potential bug on their side.

Cheers

jmreicha commented 8 years ago

Just ran in to this issue, the sleep 20 statement mentioned in https://github.com/adlogix/docker-machine-nfs/issues/31 fixed this. It would be nice to have this as a workaround until docker-machine folks fix the bug.

mxl commented 8 years ago

I also had this issue and the reason was "Block incoming connections" for nfsd in Preferences -> Security & Privacy" -> Firewall -> Firewall Options.... I have also unchecked option "Automatically allow signed software to receive incoming connections" there and I think that because of that OS X automatically blocks incoming connections for all software by setting "Block incoming connections", even signed. So you need to manually select "Allow incoming connections" for each blocked software.

brahama commented 6 years ago

Mmmm.. I still have same issue. Is this supposed to work with the new docker-machine?

docker-machine version 0.13.0, build 9ba6da9

tonivdv commented 6 years ago

Hi @brahama ,

I've got same machine version and I've got no issues at all ...

$ dm -v
docker-machine version 0.13.0, build 9ba6da9
bjoernmayer commented 6 years ago

I ran into this issue today =/ $ docker -v Docker version 18.02.0-ce, build fc4de44

$ docker-machine -v docker-machine version 0.13.0, build 9ba6da9

€dit: I was able to get it running. The /etc/hosts file on the host machine was missing 127.0.0.1 localhosts Deeplink to solution was: https://github.com/adlogix/docker-machine-nfs/issues/27#issuecomment-167681482