10GB networking fails to pass tap device to vm guests

It appears that network card properties are not been passed to vms properly.. or the virtual switch may not be working properly.

for example:

HOST

ix1: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9000 options=4a538b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,NOMAP> ether xxxxxxxxxx inet 10.100.10.21 netmask 0xffffff00 broadcast 10.100.10.255 media: Ethernet autoselect (10Gbase-T <full-duplex,rxpause,txpause>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

--cut--

vm-public: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000 ether xxxxxxxxx id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: ix1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 4 priority 128 path cost 2000 member: tap1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 9 priority 128 path cost 2000000 member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 8 priority 128 path cost 2000000 groups: bridge vm-switch viid-4c918@ nd6 options=9<PERFORMNUD,IFDISABLED>

--- cut ---

tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9000 description: vmnet/VMNAME/0/public options=80000 ether xxxxxxxxxxx groups: tap vm-port media: Ethernet autoselect status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> Opened by PID 85147

as detailed the bridge, nic and tap devices are all running properly..

the nic its self is dual port .. when connecting directly to ix0 .. I can transfer data at over 1000 MB/s as soon as I try and access the VM .. that drops to 80-100 MB/s

2: enp0s5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether xxxxxxxxx brd ff:ff:ff:ff:ff:ff inet xxxxxxxxxxx brd xxxxxxxx scope global enp0s5 valid_lft forever preferred_lft forever inet6 xxxxxxxx /64 scope link valid_lft forever preferred_lft forever

MTU and line link speed is not been set up in the guest vm properly.. on the virtual switch .. it is bound directly to ix1.. this nic is dedicated to that switch and has nothing else on it..

yet on a pure 10gb network using jumbo frames and 9kmtu .. the fastest speed possible is 114 MB/s

but on the exact same host.. I can copy files a multitude of ways and get over 1000 MB/s

according to several posts on the freebsd forums and other places .. many people are haveing the same problem .. (10 gb is not working)

other vm notes: root@vm:~# dmesg | grep enp [ 2.127832] virtio_net virtio2 enp0s5: renamed from eth0

root@vm:~# mii-tool -v enp0s5 SIOCGMIIPHY on 'enp0s5' failed: Operation not supported

root@vm:~# ethtool enp0s5 Settings for enp0s5: Supported ports: [ ] Supported link modes: Not reported Supported pause frame use: No Supports auto-negotiation: No Supported FEC modes: Not reported Advertised link modes: Not reported Advertised pause frame use: No Advertised auto-negotiation: Nocat /sys Advertised FEC modes: Not reported Speed: Unknown! Duplex: Unknown! (255) Port: Other PHYAD: 0 Transceiver: internal Auto-negotiation: off Link detected: yes

root@vm:~# lshw -C network -network description: Ethernet controller product: Virtio network device vendor: Red Hat, Inc. physical id: 5 bus info: pci@0000:00:05.0 version: 00 width: 32 bits clock: 33MHz capabilities: msix msi bus_master cap_list rom configuration: driver=virtio-pci latency=64 resources: irq:18 ioport:2100(size=64) memory:c0004000-c0005fff memory:c0007000-c00077ff -virtio2 description: Ethernet interface physical id: 0 bus info: virtio@2 logical name: enp0s5 serial: xxxxxxxxxxx capabilities: ethernet physical configuration: autonegotiation=off broadcast=yes driver=virtio_net driverversion=1.0.0 ip=1.1.1.1 link=yes multicast=yes

as you can see there is no indication of network link speed.

churchers / vm-bhyve

10GB networking fails to pass tap device to vm guests #497