search

virtio-win / kvm-guest-drivers-windows

Windows paravirtualized drivers for QEMU\KVM
https://www.linux-kvm.org/page/WindowsGuestDrivers
BSD 3-Clause "New" or "Revised" License
1.99k stars 382 forks source link

virtio-net is completely broken in terms of LSO / RSO #1131

Closed TripleEmcoder closed 2 weeks ago

TripleEmcoder commented 3 weeks ago

(or I am very bad at configuring KVM / QEMU / Windows, but I am using the defaults mostly for the repro here)

Describe the bug To get acceptable TX performance in a Windows machine using a virtio NIC one must disable Large Send Offload in the adapter properties. This increases TX speed from ca. 1Mbps to 2Gbps and affects both guest-to-host, guest-to-guest traffic. There are numerous reports of this on the internet, I can probably link a few threads if needed.

To get acceptable performance when on the RX-side of transfers coming in from other guests one must disable TCP Segment Offload on those machines - or perhaps there is a way to do it on the vnetX interface on the host, but I haven't discovered one. This increases transfers to Windows to 3Gbps from the original ca. 15Mbps when Linux transmits or 100Kbps (sic!) when data originates from FreeBSD.   To Reproduce

  1. For LSO create a Windows VM with macvtap or bridge virtio device, install drivers, measure with iperf(3).
  2. For TSO create Windows+Linux or Windows+FreeBSD machines, create a bridge, connect virtio NICs to bridge, assign IPs, measure with iperf(3).

Expected behavior On Windows virtio NICs should provide workable speeds in their default configuration (a few Gbps feels adequate). Currently in the default setup speeds allow SSH sessions, but SMB file transfers from TrueNAS (FreeBSD) fail completely, same any attempt to access its web UI.

Ideally Windows NICs would provide the similar performance to virtio on Linux, which provides raw speeds in the 20Gbps range both for host+guest and guest+guest traffic in both directions, which requires working LSO and RSO it seems.

The above has been tested on:

Host:

VM:

Additional context

Full table of measurements:

v--(-c) TX / RX (-s)--> Fedora Windows TrueNAS Ubuntu
Fedora Workstation 40 (Host) 23G / 20G 13G / 2G* 23G / 17G 22G / 20G
Windows 11 2G* / 14G 11G / 12G 3.5G / ~0 4G* / 20M
TrueNAS (FreeBSD 13) 18G / 26G ~0 / 3.5G 33G / 31G 16G / 12G
Ubuntu 22.04 21G / 21G 15M / 4G* 12G / 15G 20G / 17G

For the example cell of 3.5G / ~0 the table reads as follows: when running iperf3 -s on TrueNAS and iperf3 -c <IP> on Windows 11 you get 3.5Gbps of outgoing traffic and then with --reverse the transfer is basically stalled (measured 50-150Kbps).

To see the basiline capabilities of the IPv4 stack I also measeured in-VM localhost-loopback speeds.

There are similar values on the other side of the diagonal for when -s and -c <IP> were called in reverse, to see if the socket setups have any influence on the values.  

ifconfig vnet1 -tso v--(-c) TX / RX (-s)--> Fedora Windows TrueNAS Ubuntu
TrueNAS (FreeBSD 13) 2G / 26G 3G / 3.5G 33G / 31G 4.5G / 12G
ethtool -K enp7s0 tso off v--(-c) TX / RX (-s)--> Fedora Windows TrueNAS Ubuntu
Ubuntu 22.04 2G / 21G 3G / 4G 4G / 15G 20G / 17G
brotkastn commented 3 weeks ago

I think i can confirm this bug. However, i suspect this might be due to a newer Windows (or host) upgrade breaking things.

My wife uses a Windows VM on our "NAS", where she puts all her photos. The resulting network-performance was fine, reading new photos and exporting finished ones (or the backup of the library) was pleasantly fast and the limiting factor have been the disks.

However, i went ahead and upgraded Windows to 11 and upgraded the host from 6.8.1 to 6.10.2. Since then, importing new photos is very slow, the library backup gets cancelled because of network errors. RX seems to be better than TX, in other Words.

Currently i am running

Host:

Disto: Gentoo
Kernel version: 6.10.4
QEMU version: 8.2.3
QEMU command line: I can provide it if necessary, but really these were just VMs created with the virt-manager wizard and don't contain anything out of the ordinary
libvirt version: 10.3.0
libvirt XML file: as above

VM:

Windows version: 11 23H2
Which driver has a problem: virtio-net
Driver version: 0.1.262

Since this upgrade, especially TX is problematic. If one uses wireshark to look at the network-traffic caused by the VM - with all acceleration features enabled - i get frequent "duplicate ACKs" and "Out of order" errors. My current test-file was a 2GB .zip and i was unable to copy it onto the nas (TX if seen from the Windows 11 VM) without windows cancelling the transfer because of errors.

Currently, i have disabled every offloading feature of the driver using the windows device properties, which seems to have fixed the problem. Since this was working at one point in time, there seems to be a regression somewhere. If i throw a LinuxLive .iso in the CD-ROM drive of the VM and rerun the tests i get good performance, stable connections and wireshark does not complain.

If i can provide anything to help to narrow that problem down, i am happy to help/test/...

Thank you for your work on the virtio-drivers, have a nice day, Michael

ybendito commented 3 weeks ago

Can you please attach complete libvirt XML or complete qemu command line?

YanVugenfirer commented 3 weeks ago

Please also run "ethtool -k" on the host and the target machine

pengu1981 commented 3 weeks ago

I can confirm this issue on the following System:

QEMU Version: 9.0.2 libvirt Version: 10.5.0 virt-manager Version: 4.1.0

Host: Gentoo Linux, Kernel 6.10.4

Guest OS(es): Windows 10 and 11 with latest updates using latest VirtIO drivers (0.1.262-2) TrueNAS Scale 24.04

All guests are connected to the same bridge ( "br1" no hardware device attached)

ethtool -K br1 tso off or ethtool -K vnet(x) tso off fixes the issue.

No modifications on any guest needed.

TripleEmcoder commented 3 weeks ago

Windows VM:

<domain type="kvm">
  <name>win10</name>
  <uuid>ca6d862f-1a43-4106-96bd-45d444819735</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://microsoft.com/win/10"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit="KiB">8388608</memory>
  <currentMemory unit="KiB">8388608</currentMemory>
  <memoryBacking>
    <hugepages/>
    <nosharepages/>
    <locked/>
  </memoryBacking>
  <vcpu placement="static">2</vcpu>
  <os>
    <type arch="x86_64" machine="pc-q35-8.2">hvm</type>
    <boot dev="hd"/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode="custom">
      <relaxed state="on"/>
      <vapic state="on"/>
      <spinlocks state="on" retries="8191"/>
    </hyperv>
    <vmport state="off"/>
  </features>
  <cpu mode="host-passthrough" check="none" migratable="on"/>
  <clock offset="localtime">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
    <timer name="hypervclock" present="yes"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type="file" device="cdrom">
      <driver name="qemu" type="raw"/>
      <source file="/mnt/ssd/imgs/Win10_English_x64.iso"/>
      <target dev="sdb" bus="sata"/>
      <readonly/>
      <address type="drive" controller="0" bus="0" target="0" unit="1"/>
    </disk>
    <disk type="file" device="cdrom">
      <driver name="qemu" type="raw"/>
      <source file="/mnt/ssd/imgs/virtio-win-0.1.262.iso"/>
      <target dev="sdc" bus="sata"/>
      <readonly/>
      <address type="drive" controller="0" bus="0" target="0" unit="2"/>
    </disk>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/mnt/nvme/kvm/win10.qcow2"/>
      <target dev="vda" bus="virtio"/>
      <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/>
    </disk>
    <controller type="usb" index="0" model="qemu-xhci" ports="15">
      <address type="pci" domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
    </controller>
    <controller type="pci" index="0" model="pcie-root"/>
    <controller type="pci" index="1" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="1" port="0x10"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="2" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="2" port="0x11"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x1"/>
    </controller>
    <controller type="pci" index="3" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="3" port="0x12"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x2"/>
    </controller>
    <controller type="pci" index="4" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="4" port="0x13"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x3"/>
    </controller>
    <controller type="pci" index="5" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="5" port="0x14"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x4"/>
    </controller>
    <controller type="pci" index="6" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="6" port="0x15"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x5"/>
    </controller>
    <controller type="pci" index="7" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="7" port="0x16"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x6"/>
    </controller>
    <controller type="pci" index="8" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="8" port="0x17"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x7"/>
    </controller>
    <controller type="pci" index="9" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="9" port="0x18"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="10" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="10" port="0x19"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x1"/>
    </controller>
    <controller type="pci" index="11" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="11" port="0x1a"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x2"/>
    </controller>
    <controller type="pci" index="12" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="12" port="0x1b"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x3"/>
    </controller>
    <controller type="pci" index="13" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="13" port="0x1c"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x4"/>
    </controller>
    <controller type="pci" index="14" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="14" port="0x1d"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x5"/>
    </controller>
    <controller type="sata" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1f" function="0x2"/>
    </controller>
    <controller type="virtio-serial" index="0">
      <address type="pci" domain="0x0000" bus="0x03" slot="0x00" function="0x0"/>
    </controller>
    <interface type="direct">
      <mac address="52:54:00:2f:7f:ff"/>
      <source dev="enp4s0" mode="bridge"/>
      <model type="virtio"/>
      <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
    </interface>
    <interface type="bridge">
      <mac address="52:54:00:85:fc:71"/>
      <source bridge="bridge0"/>
      <model type="virtio"/>
      <driver name="vhost" txmode="iothread" ioeventfd="on" event_idx="on" queues="12">
        <host csum="on" gso="on" tso4="on" tso6="on" ecn="on" ufo="on" mrg_rxbuf="on"/>
        <guest csum="on" tso4="on" tso6="on" ecn="on" ufo="on"/>
      </driver>
      <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/>
    </interface>
    <serial type="pty">
      <target type="isa-serial" port="0">
        <model name="isa-serial"/>
      </target>
    </serial>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
    <channel type="spicevmc">
      <target type="virtio" name="com.redhat.spice.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="1"/>
    </channel>
    <input type="tablet" bus="usb">
      <address type="usb" bus="0" port="1"/>
    </input>
    <input type="mouse" bus="ps2"/>
    <input type="keyboard" bus="ps2"/>
    <graphics type="spice" autoport="yes">
      <listen type="address"/>
      <image compression="off"/>
    </graphics>
    <sound model="ich9">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1b" function="0x0"/>
    </sound>
    <audio id="1" type="spice"/>
    <video>
      <model type="virtio" heads="1" primary="yes"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>
    </video>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="2"/>
    </redirdev>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="3"/>
    </redirdev>
    <watchdog model="itco" action="reset"/>
    <memballoon model="virtio">
      <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/>
    </memballoon>
  </devices>
</domain>

Linux VM:

<domain type="kvm">
  <name>OSIRIS</name>
  <uuid>75cdf1f7-d812-47b2-9424-d3229e7e8f8f</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://ubuntu.com/ubuntu/22.04"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit="KiB">16777216</memory>
  <currentMemory unit="KiB">16777216</currentMemory>
  <memoryBacking>
    <hugepages/>
    <nosharepages/>
    <locked/>
    <source type="memfd"/>
    <access mode="shared"/>
  </memoryBacking>
  <vcpu placement="static">4</vcpu>
  <os firmware="efi">
    <type arch="x86_64" machine="pc-q35-8.2">hvm</type>
    <firmware>
      <feature enabled="yes" name="enrolled-keys"/>
      <feature enabled="yes" name="secure-boot"/>
    </firmware>
    <loader readonly="yes" secure="yes" type="pflash" format="qcow2">/usr/share/edk2/ovmf/OVMF_CODE_4M.secboot.qcow2</loader>
    <nvram template="/usr/share/edk2/ovmf/OVMF_VARS_4M.secboot.qcow2" format="qcow2">/var/lib/libvirt/qemu/nvram/OSIRIS_VARS.qcow2</nvram>
    <boot dev="hd"/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <vmport state="off"/>
    <smm state="on"/>
  </features>
  <cpu mode="host-passthrough" check="none" migratable="on"/>
  <clock offset="utc">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/mnt/nvme/vms/OSIRIS.qcow2"/>
      <target dev="vda" bus="virtio"/>
      <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/>
    </disk>
    <controller type="usb" index="0" model="qemu-xhci" ports="15">
      <address type="pci" domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
    </controller>
    <controller type="pci" index="0" model="pcie-root"/>
    <controller type="pci" index="1" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="1" port="0x10"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="2" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="2" port="0x11"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x1"/>
    </controller>
    <controller type="pci" index="3" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="3" port="0x12"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x2"/>
    </controller>
    <controller type="pci" index="4" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="4" port="0x13"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x3"/>
    </controller>
    <controller type="pci" index="5" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="5" port="0x14"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x4"/>
    </controller>
    <controller type="pci" index="6" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="6" port="0x15"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x5"/>
    </controller>
    <controller type="pci" index="7" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="7" port="0x16"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x6"/>
    </controller>
    <controller type="pci" index="8" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="8" port="0x17"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x7"/>
    </controller>
    <controller type="pci" index="9" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="9" port="0x18"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="10" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="10" port="0x19"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x1"/>
    </controller>
    <controller type="pci" index="11" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="11" port="0x1a"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x2"/>
    </controller>
    <controller type="pci" index="12" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="12" port="0x1b"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x3"/>
    </controller>
    <controller type="pci" index="13" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="13" port="0x1c"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x4"/>
    </controller>
    <controller type="pci" index="14" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="14" port="0x1d"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x5"/>
    </controller>
    <controller type="sata" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1f" function="0x2"/>
    </controller>
    <controller type="virtio-serial" index="0">
      <address type="pci" domain="0x0000" bus="0x03" slot="0x00" function="0x0"/>
    </controller>
    <controller type="scsi" index="0" model="virtio-scsi">
      <address type="pci" domain="0x0000" bus="0x09" slot="0x00" function="0x0"/>
    </controller>
    <filesystem type="mount" accessmode="passthrough">
      <driver type="virtiofs"/>
      <source dir="/home/marcin"/>
      <target dir="DRAGONFLY marcin"/>
      <address type="pci" domain="0x0000" bus="0x08" slot="0x00" function="0x0"/>
    </filesystem>
    <interface type="direct">
      <mac address="00:0c:29:04:1e:11"/>
      <source dev="enp4s0" mode="bridge"/>
      <model type="virtio"/>
      <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
    </interface>
    <interface type="bridge">
      <mac address="00:0c:29:04:1e:1b"/>
      <source bridge="bridge0"/>
      <model type="virtio"/>
      <address type="pci" domain="0x0000" bus="0x07" slot="0x00" function="0x0"/>
    </interface>
    <serial type="pty">
      <target type="isa-serial" port="0">
        <model name="isa-serial"/>
      </target>
    </serial>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
    <channel type="unix">
      <target type="virtio" name="org.qemu.guest_agent.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="1"/>
    </channel>
    <channel type="spicevmc">
      <target type="virtio" name="com.redhat.spice.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="2"/>
    </channel>
    <input type="tablet" bus="usb">
      <address type="usb" bus="0" port="1"/>
    </input>
    <input type="mouse" bus="ps2"/>
    <input type="keyboard" bus="ps2"/>
    <graphics type="spice" autoport="yes">
      <listen type="address"/>
      <image compression="off"/>
    </graphics>
    <sound model="ich9">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1b" function="0x0"/>
    </sound>
    <audio id="1" type="spice"/>
    <video>
      <model type="qxl" ram="65536" vram="65536" vgamem="16384" heads="1" primary="yes"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>
    </video>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="2"/>
    </redirdev>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="3"/>
    </redirdev>
    <watchdog model="itco" action="reset"/>
    <memballoon model="virtio">
      <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/>
    </memballoon>
    <rng model="virtio">
      <backend model="random">/dev/urandom</backend>
      <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/>
    </rng>
  </devices>
</domain>

Here's ethtool -k output with only the Windows LSO=off workaround implemented, i.e. when WIndows -> Linux gives 3Gbps throughput and Linux -> Windows 30Mbps.

EDIT: Previously I only included Linux guest interface options, now I also added Windows vnet data. EDIT2: I am actually so accustomed now to the Windows LSO=off fix that I forgot the dumps do include this workaround. Updated the description accordingly.

anthonyde commented 2 weeks ago

A bug was fixed in 6.10.5 that looks like this. It was introduced in 6.10.3.

https://lore.kernel.org/all/20240729201108.1615114-1-willemdebruijn.kernel@gmail.com/

YanVugenfirer commented 2 weeks ago

@anthonyde Thanks!

@TripleEmcoder @brotkastn - can you test with the updated kernel?

TripleEmcoder commented 2 weeks ago

Yes, thank you! I am running 6.10.6 at the moment and Linux/FreeBSD -> Windows with TSO on now provides speeds in the 7-11 Gbps range, which means at least a 2x improvement over TSO off and infinitely more than with the bug before. Didn't think to look for kernel bugs, this is quite a complicated pipeline overall for an end/advanced-user.

Same improvements for the reverse direction, Windows -> Linux with LSO on gives me 11-15 Gbps, instead of 1-3 Gbps with LSO off and instead of Mbps before. Interestingly the report of people having this particular problem found on the internet were older than the 6.10.3 kernel.

pengu1981 commented 2 weeks ago

I can confirm that this issue is gone when running 6.10.6.

copying files from a Windows 10 / 11 host to a TNS host works as espected copying files from TNS host to a windows 10 / 11 Host is also working with expected speeds

The second issue, that the copy process from Windows 10 or 11 Host to the TNS host didn't start according to the Windows copy dialog is also gone.

Exactly the same as disabling TSO on "br1" or the Windows hosts vnet* device in earlier kernels.

ybendito commented 2 weeks ago

@TripleEmcoder Please feel free to close the issue if it's not actual anymore

brotkastn commented 2 weeks ago

@anthonyde Thanks!

@TripleEmcoder @brotkastn - can you test with the updated kernel?

I can also confirm that upgrading to 6.10.6 fixes the issue here. Thank you everyone who helped to track this down :)