nat can end up in `round-robin` state

igalic commented 7 years ago

When configuring a jailhost on digitalocean, its network may end up looking like this

root@meenix:~ # pfctl -s nat
nat on vtnet0 inet from 10.0.0.2 to any -> (vtnet0) round-robin
…

This will cause the jails' networking to be an order of magnitude slower, than on the body. I don't know what this is caused by; it could have something to do with digitalocean's default private network:

root@meenix:~ # ifconfig
vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 3a:0c:c6:47:2f:12
        inet6 fe80::380c:c6ff:fe47:2f12%vtnet0 prefixlen 64 scopeid 0x1 
        inet6 2a03:b0c0:3:d0::3585:b001 prefixlen 64 
        inet 207.154.239.110 netmask 0xfffff000 broadcast 207.154.239.255 
        inet 10.19.0.6 netmask 0xffff0000 broadcast 10.19.255.255 
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T <full-duplex>
        status: active
vtnet1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 1e:4f:85:54:53:2c
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T <full-duplex>
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128 
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 
        inet 127.0.0.1 netmask 0xff000000 
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo 
lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 10.0.0.2 netmask 0xffffffff 
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo 
root@meenix:~ # netstat -nr
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            207.154.224.1      UGS      vtnet0
10.0.0.2           link#4             UH          lo1
10.19.0.0/16       link#1             U        vtnet0
10.19.0.6          link#1             UHS         lo0
127.0.0.1          link#3             UH          lo0
207.154.224.0/20   link#1             U        vtnet0
207.154.239.110    link#1             UHS         lo0

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
default                           2a03:b0c0:3:d0::1             UGS      vtnet0
::1                               link#3                        UH          lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
2a03:b0c0:3:d0::/64               link#1                        U        vtnet0
2a03:b0c0:3:d0::3585:b001         link#1                        UHS         lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%vtnet0/64                  link#1                        U        vtnet0
fe80::380c:c6ff:fe47:2f12%vtnet0  link#1                        UHS         lo0
fe80::%lo0/64                     link#3                        U           lo0
fe80::1%lo0                       link#3                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0
root@meenix:~ #

i changed the nat rule to nat on vtnet0 from lo1:network to any -> (vtnet0:0) and this made it worked!

i don't know if changing the pf.conf template to:

{% for network in pf_nat_jail_networks %}
nat on {{ pf_nat_interface }} from {{ network }} to any -> ({{ pf_nat_interface }}:0)
{% endfor %}

would make it work for everyone.

fschulze commented 7 years ago

I've tried it on my NAS and it seems to work fine. Will test further.

fschulze commented 7 years ago

FTR, the command to see if round-robin is active is pfctl -s nat

fschulze commented 7 years ago

It seems like this also affects servers with multiple IP4 adresses. @tomster this seems to fix our backup server issue from today.

fschulze commented 6 years ago

Released with 2.3.0

ployground / bsdploy

nat can end up in `round-robin` state #102