Is it possble to share wireless internet on the host?

[rodyaj]

Is it possible to use NAT to allow guests to share the host wireless internet? I followed the instructions to enable the NAT with 'vm switch nat public on' and also started dnsmasq service, but DHCP did not resolve inside the VMs I tested. I did not add the physical wlan0 to the bridge, as I've read this will not work. Thanks for your time.

[churchers]

Hmm, I've not done particularly extensive testing with the NAT functionality as I rarely use NAT in my own bhyve deployments, but I can't see why it wouldn't be at least assigning DHCP to the guest.

Can you output the bridge details from ifconfig when the guest is running, the dnsmasq configuration (/usr/local/etc/dnsmasq.conf). and also the PF config from $vm_dir/.config/pf-nat.conf I'll see if I can try and replicate the problem.

I can't see any reason why it shouldn't be possible for this to work if your default gateway is via the wireless.

But yes, don't add the wireless interface to the bridge. In NAT configuration, the bridge is effectively a switch on the private LAN, and the wireless interface should be on the 'public' side of the NAT. The FreeBSD host is responsible for taking packets from the private LAN, and forwarding them out to the wireless network (which is where pf comes in).

[rodyaj]

ifconfig bridge0:

bridge0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: vm-public
        ether 02:38:d5:af:42:00
        nd6 options=1<PERFORMNUD>
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 7 priority 128 path cost 2000000

/usr/local/etc/dnsmasq.conf:

# vm-bhyve dhcp
port=0
domain-needed
no-resolv
except-interface=lo0
bind-interfaces
local-service
dhcp-authoritative

interface=bridge0
dhcp-range=172.16.0.10,172.16.0.254

$vm_dir/.config/pf-nat.conf:

# vm-bhyve nat
nat on wlan0 from {172.16.0.0/24} to any -> (wlan0)

Thanks for the help.

[churchers]

Hmm well the first problem I can see is that it doesn't seem to have assigned the 172.16.0.1 address to the bridge. You don't happen to be using the same 172.16.0.x range on your local network do you?

I'll do some more testing of the NAT functions in a similar configuration, although I won't be able to do this till tomorrow at the earliest.

[rodyaj]

Well I thought that bridge0 output wasn't what I remembered. This was a fresh install of vm-bhyve and, for whatever reason, rebooting assigned the IP:

bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: vm-public
        ether 02:38:d5:af:42:00
        inet 172.16.0.1 netmask 0xffffff00 broadcast 172.16.0.255 
        nd6 options=1<PERFORMNUD>
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 7 priority 128 path cost 2000000

Still the same problem as before though. I don't use any of those IPs on my local network.

[churchers]

Is the guest already installed or are you trying to get DHCP in the installer? If possible try assigning a manual IP to the guest so we can at least see if the problem is just DHCP, or if NAT isn't working as well.

You can use 172.16.0.2/255.255.255.0 with a gateway of 172.16.0.1.

[rodyaj]

I already tried assigning static IP, both in the installer and in an installed VM, and I can't ping anything, so it seems the NAT isn't working either. I've been using the Ubuntu example template for the VM (if it means anything):

guest="ubuntu"
cpu=1
memory=512M
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"

[churchers]

Hmm looks like there's a few things going wrong but it's not obvious why. Are you able to at least ping the IP address of the bridge from inside the guest? Do you have any existing pf configuration that may be blocking traffic between the guest/host? (If you do have existing pf config it may be worth trying with just the vm-bhyve rules first just to rule out the firewall blocking anything)

[rodyaj]

There were some pf rules autogenerated I made and forgot about for iocage jails in /etc/pf.conf that must've conflicted with the sourced vm-bhyve pf config. DHCP works fine now. I'll try to isolate the exact rules that conflict today and post the details, in case anyone else gets stuck with a similar problem. Just thought I'd let you know, so you don't waste any more time. Thanks for the help.

[churchers]

Hi @rodyaj Great that you managed to get it working and that the vm-bhyve part seems to have been working as it should. The problem with NAT is that it adds quite a bit of extra complication and it's not something I use often on FreeBSD.

I did test Ubuntu with NAT this morning without any issues although I have no BSD machines using wireless. I can't see wireless being a problem in a NAT set up though, but I have heard that it can be a problem in a bridged configuration.

[rodyaj]

I thought I'd solved everything because DHCP assigns an IP now, but I still can't resolve any addresses. The only thing I can ping in the VM is the bridge0 IP of 172.16.0.1. I've tried using a static IP setup too. Do I need to add some nameservers in resolv.conf in the VM as well? I tried using the gateway IP as the nameserver, as DHCP didn't generate anything.

Output of ifconfig in the VM:

eth0      Link encap:Ethernet  HWaddr 00:a0:98:b0:3a:40  
          inet addr:172.16.0.178  Bcast:172.16.0.255  Mask:255.255.255.0
          inet6 addr: fe80::2a0:98ff:feb0:3a40/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:384 (384.0 B)  TX bytes:1110 (1.1 KB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:28 errors:0 dropped:0 overruns:0 frame:0
          TX packets:28 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:2072 (2.0 KB)  TX bytes:2072 (2.0 KB)

[churchers]

Yes you'll need some DNS servers in resolv.conf on the guest. I usually just use Google's. Dnsmasq could probably be used to provide DNS but I've not even thought to look into it.

[rodyaj]

I've tried the Google nameservers, but when I ping an address ping hangs and eventually says 'unknown host'.

[churchers]

Can you ping Google's name server on 8.8.8.8 by IP address? If not can you confirm the guest has picked up a gateway of 172.16.0.1 and there are no other firewall rules on the host that might be blocking Internet access.

[rodyaj]

I can't ping Google nameservers.

Running route on the guest:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         172.16.0.1      0.0.0.0         UG    0      0        0 eth0
172.16.0.0      *               255.255.255.0   U     0      0        0 eth0

pf nat rules on the host (pfctl -sn):

nat on wlan0 inet from 172.16.0.0/24 to any -> (wlan0) round-robin

pf firewall rules (pfctl -sr) is blank.

ipfw is disabled, so I can't think of any other conflicting rules.

[churchers]

Can you try the following NAT rule from the bhyve wiki:

nat on wlan0 from bridge0:network to any -> (bridge0)

Change the rule in .config/pf-nat.conf then restart the pf service. I know practically nothing about pf so I'm not sure of the significance of the interface name at the end, but this is listed as a solution for using bhyve guests with wireless, which suggests this should at least be possible. If it works I'll test on my non-wireless test machine and update the vm-bhyve code if it looks good. It's just strange that the existing rule works perfectly on my test machine.

[rodyaj]

I'm still having the same problems with the new rule. So you have tested this with wireless and it worked for you?

[rodyaj]

In case it is useful, here is my entire ifconfig output on the host:

iwn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 2290
    ether e0:9d:31:0b:e2:e0
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    media: IEEE 802.11 Wireless Ethernet autoselect mode 11ng
    status: associated
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128 
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 
    inet 127.0.0.1 netmask 0xff000000 
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether e0:9d:31:0b:e2:e0
    inet6 fe80::e29d:31ff:fe0b:e2e0%wlan0 prefixlen 64 scopeid 0x3 
    inet 192.168.1.67 netmask 0xffffff00 broadcast 192.168.1.255 
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    media: IEEE 802.11 Wireless Ethernet MCS mode 11ng
    status: associated
    ssid PlusnetWireless94CBB7 channel 11 (2462 MHz 11g ht/20) bssid 9c:97:26:94:cb:b7
    country US authmode WPA2/802.11i privacy ON deftxkey UNDEF
    TKIP 2:128-bit TKIP 3:128-bit powersavemode CAM powersavesleep 100
    txpower 11 bmiss 10 scanvalid 60 bgscan bgscanintvl 300 bgscanidle 250
    roam:rssi 7 roam:rate 64 protmode CTS ampdulimit 64k ampdudensity 8
    -amsdutx amsdurx shortgi wme roaming MANUAL
pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33160
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: vm-public
    ether 02:38:d5:af:42:00
    inet 172.16.0.1 netmask 0xffffff00 broadcast 172.16.0.255 
    nd6 options=1<PERFORMNUD>
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 6 priority 128 path cost 2000000
tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: vmnet-ubuntuvm-0-public
    options=80000<LINKSTATE>
    ether 00:bd:0a:50:00:00
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    media: Ethernet autoselect
    status: active
    Opened by PID 5621

[churchers]

No I have nothing wireless to test with which is half the problem. If I could sit down with it and run tests I could probably get it working. It's just that rule is listed in the official bhyve wiki as a way of getting so guests to use a wireless uplink, so you'd think it would work.

[rodyaj]

I'm trying to think of other things that could conflict with it, but they are the only firewall rules I have running now. It is a PC-BSD system and not FreeBSD I'm testing on, but I don't think that would contribute much significance to this problem, other than making sure ipfw is disabled.

[rodyaj]

I noticed from my ifconfig output that the tap is not assigned an IP. Do I need to assign it one with ifconfig?

[churchers]

I'm really stumped at the moment. The NAT rule looks ok, and is apparently suggested for this exact situation. Even without anything else working that should at least allow you to ping the upstream routers IP address and the Internet unless something else is affecting pf.

[churchers]

The tap interface is tied to eth0 in the guest and has the address assigned inside the guest.

[rodyaj]

I'm stuck myself for now. Might try another day when I have a fresh head. I need to try and understand what is going on better under-the-hood e.g., with the dnsmasq options before I can fix it.

[churchers]

If the guest is getting DHCP and the correct gateway, which it seems to be, you can pretty much ignore dnsmasq, as that's all it's used for. The problem all seems to be related to the NAT which is down to pf.

[rodyaj]

So if I assign a static IP and give it the gateway myself in the guest, does that mean I don't need to bother with dnsmasq?

[churchers]

Yes. Dnsmasq only does the DHCP. It isn't strictly required.

[rodyaj]

Just wanted to say that the pf config and devices generated by 'vm switch create public' does work afterall. I don't know why it wasn't working before. Perhaps preexisting bridge and tap settings in my /etc/rc.conf were preventing the devices from initialising and/or binding to each other properly. With a clean /etc/rc.conf everything works.

[jamesamcl]

Dnsmasq could probably be used to provide DNS but I've not even thought to look into it.

I added to my dnsmasq.conf:

dhcp-option=6,10.8.232.199,10.8.232.109

where 10.8.232.199 and 10.8.232.109 are alternate DNS servers on my network. dnsmasq now provides those IPs over DHCP and the (linux) guest correctly generates resolv.conf with 10.8.232.199 and 10.8.232.109 as nameservers.