hashicorp / vagrant

Vagrant is a tool for building and distributing development environments.
https://www.vagrantup.com
Other
26.17k stars 4.43k forks source link

On Ubuntu 18.04, public_network mode creates two default routes and use_dhcp_assigned_default_route has no effect #10731

Open interfrastic opened 5 years ago

interfrastic commented 5 years ago

Vagrant version

Vagrant 2.2.4

Host operating system

macOS Mojave Version 10.14.3

Guest operating system

Ubuntu Bionic Beaver 18.04.2 LTS from box bento/ubuntu-18.04

Vagrantfile

Vagrant.configure("2") do |config|
  config.vm.box = "bento/ubuntu-18.04"
  config.vm.network "public_network"
end

Debug output

vagrant.log

Expected behavior

When use_dhcp_assigned_default_route is true, there should be exactly one default route, and it should be through the bridged interface eth1, like this:

vagrant@vagrant:~$ ip route
default via 147.11.4.1 dev eth1 
10.0.2.0/24 dev eth0  proto kernel  scope link  src 10.0.2.15 
147.11.4.0/22 dev eth1  proto kernel  scope link  src 147.11.4.34 

When use_dhcp_assigned_default_route is false, there should be exactly one default route, and it should be through the NAT interface eth0, like this:

vagrant@vagrant:~$ ip route
default via 10.0.2.2 dev eth0 
10.0.2.0/24 dev eth0  proto kernel  scope link  src 10.0.2.15 
147.11.4.0/22 dev eth1  proto kernel  scope link  src 147.11.4.160 

Actual behavior

Regardless of whether use_dhcp_assigned_default_route is true or false, there are two default routes, one through the NAT interface eth0 and one through the bridged interface eth1, like this:

vagrant@vagrant:~$ ip route
default via 10.0.2.2 dev eth0 proto dhcp src 10.0.2.15 metric 100 
default via 147.11.4.1 dev eth1 proto dhcp src 147.11.6.167 metric 100 
10.0.2.0/24 dev eth0 proto kernel scope link src 10.0.2.15 
10.0.2.2 dev eth0 proto dhcp scope link src 10.0.2.15 metric 100 
147.11.4.0/22 dev eth1 proto kernel scope link src 147.11.6.167 
147.11.4.1 dev eth1 proto dhcp scope link src 147.11.6.167 metric 100 

The ordering of the routes seems to vary in ways that are difficult to predict. This causes protocols like NFS, which can fail when passed through NAT, to work in some cases and fail in others, despite an identical Vagrantfile (and therefore guest OS configuration) in each case.

Steps to reproduce

  1. Create a Vagrantfile that specifies Ubuntu 18.04, or any other Debian distribution that uses Netplan for network configuration.
  2. Edit the Vagrantfile to enable public_network mode.
  3. Run vagrant up and choose to bridge to whichever network interface is connected to the Internet.
  4. Run vagrant ssh to connect to the VM.
  5. Run ip route in the VM.

Note that editing the Vagrantfile to set the use_dhcp_assigned_default_route option to true has no effect on the routing.

Editing the Vagrantfile to specify Ubuntu 16.04, or any other Debian distribution that uses net-tools for network configuration, makes the problem of dual default routes disappear; this change also restores the proper behavior of the use_dhcp_assigned_default_route option.

A glance at the source code shows that only the net-tools Debian configuration template actually takes use_dhcp_assigned_default_route into account, a change that was implemented as a fix for GH-2648; the new Netplan configuration code, implemented for GH-9134, does not seem to consider use_dhcp_assigned_default_route at all.

References

philon123 commented 1 year ago

any update on this issue? I saw another thread open on the topic of unclear default routes dating all the way back to 2014, with just some hacky scripts proposed to manuallly change routing during provisioning. But those solutions require the VM to be reprovisioned on each boot.. So that can't be the proper solution here. This issue is written very clearly and understandably, so I'm hoping for an implementation!

For context, I'm having this issue on NixOS host with Ubuntu 22.04 client on Vagrant 2.2.19 using libvirt provider