hashicorp / vagrant

Vagrant is a tool for building and distributing development environments.
https://www.vagrantup.com
Other
26.33k stars 4.44k forks source link

[NFS] vagrant up hangs while "Mounting NFS shared folders" #5802

Closed lordgordon closed 9 years ago

lordgordon commented 9 years ago

Hi there,

I'm trying to use NFS with FreeBDS yet every time I run vagrant up it hangs at the "Mounting NFS shared folders".

Output:

==> default: Attempting graceful shutdown of VM...
==> default: Clearing any previously set forwarded ports...
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
    default: Adapter 2: hostonly
==> default: Forwarding ports...
    default: 22 => 2222 (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: Warning: Connection timeout. Retrying...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Configuring and enabling network interfaces...
==> default: Exporting NFS shared folders...
==> default: Preparing to edit /etc/exports. Administrator privileges will be required...
Password:
==> default: Mounting NFS shared folders...

Vagrantfile:

# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure(2) do |config|
  config.vm.guest = :freebsd
  config.vm.box = "dev-freebsd"
  config.vm.box_check_update = false
  config.vm.network "private_network", ip: "192.168.1.2"
  config.vm.synced_folder ".", "/vagrant", create: true, :nfs => true, :mount_options => ['nolock,vers=3,tcp,noatime,clientaddr=192.168.1.2'], id: "vagrant-root"
end

/etc/exports (host):

# VAGRANT-BEGIN: 501 XXX-XXX-XXX
"/Users/mypath/myrepo" 192.168.1.2 -alldirs -mapall=501:20
# VAGRANT-END: 501 XXX-XXX-XXX

Host: OS X 10.9.5 Guest: FreeBSD 10.1-p10

Actually, the problems seems in the way vagrant runs the mount command in the guest box. While vagrant up hangs I can access the box with vagrant ssh and check the blocking commands:

ps -afxu | grep nfs
mount -t nfs -o nfsv3 192.168.1.1:/Users/mypath/myrepo /vagrant
mount_nfs -o nfsv3 192.168.1.1:/Users/mypath/myrepo

Then if I kill them and re-execute, everything works fine:

sudo killall mount mount_nfs
sudo mount -t nfs -o nfsv3 192.168.1.1:/Users/mypath/myrepo /vagrant

I'm unable to understand why the command issued by vagrant get stuck. Could you please investigate?

Thank you for your help!

Regards,

Nicholas

Further reference: https://github.com/wunki/vagrant-freebsd/issues/4

lordgordon commented 9 years ago

Sorry, my fault. I didn't notice the firewall was still enabled during vagrant up.

bronger commented 9 years ago

Because I'm suffering from the same issue currently, I wonder which firewall you mean, and why it is down when you re-execute the command manually? Thank you!

lordgordon commented 9 years ago

I have Little Snitch firewall on my host (Mac OS X). I had to temporary disable the firewall the first time to make the NFS mount.

Then, the second time, I recorded the various port used and I placed the correct rules.

cdaringe commented 9 years ago

hmmm. i confirmed my firewall is off in OSX.

cdieringer@Snapper-osx:/coins/localcoin$ sudo mount -v -t nfs -o nfsv3 192.168.1.1:/Users/cdieringer/Sublime/ /coins/test
mount_nfs: option nfsv3 deprecated, use vers=#
mount_nfs: can't mount /Users/cdieringer/Sublime/ from 192.168.1.1 onto /coins/test: Operation timed out

still searching for the cause

bronger commented 9 years ago

Maybe something in NFS is still hanging from a previous run. I solved the issue by restarting the NFS server every time before I call "vagrant up".

lordgordon commented 9 years ago

Hi there,

I no longer had the problem. I checked back my setup and I confirm the settings I already posted in this thread. To help out others, I add there further settings.

In OS X firewall settings (stealth mode is ON):

netbiosd           block incoming connections
nfsd                  allow incoming connections
rpc.lockd          allow incoming connections
rpc.rquotad      allow incoming connections
rpc.statd          allow incoming connections
rpcbind            allow incoming connections
VirtualBoxVM  allow incoming connections

In your guest OS (FreeBSD) make sure that NFS is enabled and the firewall is correctly set up. In my case I have pf and it allow everything as I use Little Snitch on OS X as global firewall.

/etc/rc.conf in FreeBSD (relevant parts to NFS and Vagrant only):

# Firewall
pf_rules="/etc/pf.conf"
pf_enable="YES"

# SSH
sshd_enable="YES"

# Virtualbox
vboxguest_enable="YES"
vboxservice_enable="YES"

# NFS
rpcbind_enable="YES"
nfs_client_enable="YES"

# extra settings
dbus_enable="NO"
avahi_daemon_enable="YES"
avahi_dnsconfd_enable="NO"
rsyncd_enable="NO"

/etc/pf.conf in FreeBSD (with a jail nat):

# Interfaces
ext_if = "em0"
int_if = "lo1"

jail_net = $int_if:network

# NAT
set skip on lo1
nat pass on $ext_if from $jail_net to any -> $ext_if
lordgordon commented 9 years ago

@bronger True: every time I changed the firewall settings I had to restart NFS.

bradfloodx commented 9 years ago

For anyone on OSX check your Firewall preferences System Preferences -> Security & Privacy -> Firewall -> Firewall Options. Make sure Automatically allow signed software to receive incoming connections is checked.

Otherwise just disable the OSX Firewall altogether if you don't need it.

lordgordon commented 9 years ago

@bradleyflood Actually, I don't have that option enabled and it works fine. In fact I don't think is a good security suggestion.

Please look at your settings, it's not necessary to disable the firewall to make NFS working properly with Vagrant.

mtpereira commented 8 years ago

Hello,

I've just tested on Yosemite and these are the daemons that I had to allow on my firewall so that the shared folder was successfully mounted:

It wasn't necessary to restart nfsd after changing the firewall in order to mount the share.

Thank you!

Thinkscape commented 8 years ago

This issue is far from resolved. It still hangs with different distros, several OSX and Vagrant versions ahead.

Here's another thing that sometimes helps:

Restart nfsd on the osx host

sudo nfsd restart

nfsd might be sensitive to power state changes, osx low-power modes, network state changes etc. Restarting it before vagrant up might help with the issue.

EvanK commented 8 years ago

I'm running into this same issue (hanging on Mounting NFS shared folders...) on OSX, and it's only started today.

I've been using vagrant (and this particular project VM with an NFS mount) for years, and the latest 1.8.1 version of vagrant since its release. The specific line from my Vagrantfile:

web.vm.synced_folder ".", "/vagrant", :nfs => true

I've tried destroying the vm, emptying the /etc/exports file, restarting nfsd and rpcbind. No joy.

In fact, the only thing that seemed to work was rebooting the entire OS...and the problem appeared again within an hour.

dacodekid commented 8 years ago

Never had this problem b4, but suddenly hit by this. It works for the first time you create / recreate a machine. Once the VM reboots, it comes back. - OSX - Vagrant 1.8.1.

EvanK commented 8 years ago

I've narrowed down my issue to something altogether different: my vms were becoming unreachable at their private ip addresses when a PIA vpn connection was established...and disconnecting it temporarily fixes the issue, so I suspect it was a network routing problem

yalcinsurkultay commented 8 years ago

@EvanK +1

It hangs when I am using VPN (Junos Pulse)

aaronjwood commented 8 years ago

@EvanK @yalcinsurkultay I've noticed that most VPN services (especially ones configured/deployed by your workplace) will by default suck up all connections in your system. For example, if you're on VPN and you spin up a Linux VM and try to SSH to it from your host, you'll see that the traffic gets taken into the VPN. I'm sure this is due to the way they're changing the routing table on your host. At least in my case the routing table gets changed and everything is sent to a different gateway.

EvanK commented 8 years ago

@aaronjwood The VPN I use from my workplace (a Cisco, iirc) doesn't have this same issue, so it really does seem to be hit or miss ¯_(ツ)

rabin-io commented 8 years ago

My 2c for this issue, Had the same problem with my Linux machine running Fedora 24 (using firewalld) and using libvirt as provider, so after reading this thread i enabled the logging option for the FW and start nmap-ing (nmap -p2049 HOST and nmap -p2049 -sU HOST ; for nfs over udp ) from inside the vm to my host (which is the nfs server), and I start seeing the traffic blocked on the virtual interface which was set to use the default zone, I changed the zone to the internal zone and enabled nfs and rpc-bind services for the internal zone, but that solved only the tcp mount option, for mouning using udp, i had to edit the service definition for nfs to include 2049/udp, and that solve my 2nd problem.

jurgenweber commented 8 years ago

I had this again, different machine.. I found I had a broken version of virtual box, VirtualBox-5.1.4 just timed out and did nothing... VirtualBox-5.1.6 works fine.

taonic commented 8 years ago

Another possible cause of the issue is to do with the network interface ordering bug reported from: https://github.com/mitchellh/vagrant/issues/7876. Fixed in: https://github.com/mitchellh/vagrant/pull/7866

bashirawaty02 commented 7 years ago

For anyone on OSX check your Firewall preferences System Preferences -> Security & Privacy -> Firewall -> Firewall Option

go to the advanced options and make sure Automatically allow signed software to receive incoming connections is checked. Or disable the OSX Firewall altogether if you don't need it.

this did work for me and its the simplest solution

chadfurman commented 7 years ago

This issue, for me, is usually caused by network connection issues between Host / Guest. If you don't have firewall issues, then make sure your host's vboxnet0 network is both up and configured with an IP.

Solution for Ubuntu 17.04:

$ sudo ip a add 192.168.33.1/255.255.255.0 dev vboxnet0
$ sudo ip link set vboxnet0 up
mpmumau commented 7 years ago

Just an FYI, @chadfurman, as I've noticed your proposed solution in a few different places, that doesn't work to solve the problem for me (hanging on "Mounting NFS shared folders...") in Arch. Not sure why Vagrant isn't working anymore; had previously worked for me for a great number of years until I started having tons of pesky little problems with it like this.

snfnwgi commented 7 years ago
default: 5986 (guest) => 55986 (host) (adapter 1)

==> default: Running 'pre-boot' VM customizations... ==> default: Booting VM... ==> default: Waiting for machine to boot. This may take a few minutes... default: WinRM address: 127.0.0.1:55985 default: WinRM username: IEUser default: WinRM execution_time_limit: PT2H default: WinRM transport: plaintext ==> default: Machine booted and ready! ==> default: Checking for guest additions in VM... ==> default: Configuring and enabling network interfaces... ==> default: Exporting NFS shared folders... ==> default: Preparing to edit /etc/exports. Administrator privileges will be required... ==> default: Mounting NFS shared folders... Vagrant attempted to execute the capability 'mount_nfs_folder' on the detect guest OS 'windows', but the guest doesn't support that capability. This capability is required for your configuration of Vagrant. Please either reconfigure Vagrant to avoid this capability or fix the issue by creating the capability.

lordgordon commented 7 years ago

@snfnwgi

Be aware that NFS is not supported on Windows hosts.

Reference: https://www.vagrantup.com/docs/synced-folders/nfs.html

snfnwgi commented 7 years ago

@lordgordon I am MAC os above windows7 operation, this can not it?

a-r-m-i-n commented 7 years ago

@lordgordon Right, but there is a vagrant plugin existing for Windows which provides NFS: https://github.com/winnfsd/vagrant-winnfsd

Unfortunately I've got the same issue, currently. Hang up on "Mounting NFS shared folders". But recently I've increased the harddisk capacity from 40GB to 500GB. Maybe this makes the mounting process incredibly slow. But this is just a guess. Update: Tested with 40GB version. So the size of the virtual hard disk is not the reason.

giacecco commented 6 years ago

A quick note to thank @rabin-io as the issue is still there on Fedora more than one year later. Other Fedora users may be suffering right now :-)

subtlegiant commented 6 years ago

For those on Fedora running into this issue I was able to resolve doing the following:

  1. Check /etc/exports for duplicate entries and remove any (I just cleared the file).
  2. Restart nfs: sudo systemctl restart nfs
  3. Try doing a vagrant up.

The problem for me was that nfs was failing due to the duplicate entry in the export file. Hope this helps.

yeongsheng-tan commented 6 years ago

@aaronjwood I confirmed VPN is the cause of nfs mount synced_folders issue. I similarly use Juno Pulse.

I tried to run vagrant up via VPN in order to access our RPM repo in our office network, at home, and the obvious routing table changes and new NIC routing interfaces created altered the behaviour of vagrant host to guest networking.

When I switched of/ disconnect from my VPN, the vagrant NFS mount worked (repeatedly tested using vagrant ver 1.9.2, 1.9.8 and 2.0.3 against Virtualbox 5.1.34). However, I can obviously no longer access our office RPM repo and that caused the provisioner to fail, which our provisioning script is written to point our CentOS7 GuestOS to an internal RPM repo.

To get around the above, I disconnect from our VPN, run vagrant up till NFS mount succeeds. At that point, I manually connect to our VPN so that the provisioner can continue to pull in the needed during GuestOS yum install of our internally managed RPM packages.

Thanks for the hint. Not the best solution, but it works and allows me to work from home while continue development.

pabrrs commented 6 years ago

I've been stuck in same question, and i solved this by disabling the firewall.

poncianodiego commented 6 years ago

sudo service nfs-kernel-server restart worked for me on Linux Ubuntu 18.04

@Thinkscape I think is kind of the same approach but for linux so thanks for the idea! :)

guih commented 5 years ago

I've narrowed down my issue to something altogether different: my vms were becoming unreachable at their private ip addresses when a PIA vpn connection was established...and disconnecting it temporarily fixes the issue, so I suspect it was a network routing problem

@EvanK There's an "Allow LAN Traffic" option if you're running the new PIA desktop client that fixes the issue.

lukejsimonetti commented 5 years ago

This happens occasionally for me. However when I disable the firewall it normally doesn't help. For me the solution is to restart the host (MacOS) and then everything boots just fine.

fooness commented 5 years ago

Can confirm this problem still exists on macOS 10.14.3 …

gideoncresswell commented 5 years ago

I had to install Sophos Home virus software on my Mac (macOS 10.14.4) for my current job as it was a VPN requirement and I had this issue. As soon as I uninstalled Sophos Home I had no vagrant issues.

Hyperadministrator commented 5 years ago

Same problem still exists on MacOS 10.14.6. I think I have tried most of solutions described here but the problem keeps coming back and back. In addition to the previously given tips doing the following has stopped the problem occurring for once or twice (or maybe it's not related at all- at least after doing this the NFS mount seems to work :) ):

1) Proven versions that have worked: VirtualBox: 5.2.22 r126460 (Qt5.6.3), 5.2.32-132073, 6.0.10 r132072 (Qt5.6.3) vagrant: 2.2.1, 2.2.2, 2.2.5

2) Restarting the workstation :)

3) If you're using VPN: Destroy VM, turn off VPN, build up VM again

4) Switch off WiFi and use cable network. Halt the VM (or even destroy and rebuild) and run vagrant up again.

Janoyan commented 4 years ago

Try vagrant up --debug

In debug mode you can see what is the problem. In my case it was with one of shared folder.

ghost commented 4 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.