OrangePi5Plus32G cannot bind RTL8125 2.5GbE to vfio-pci

[rnovikov]

root@ubuntu:/sys/bus/pci/drivers/vfio-pci# echo 0004:41:00.0 > bind bash: echo: write error: No such device

Device OrangePi5Plus32G Ubuntu 24.04

[Joshua-Riek]

Can you modprobe vfio-pci?

[rnovikov]

I had done it,before trying to bind. vfio-pci probbed successifull. Do you need some additional traces? FYI: i am trying to build and run dpdk and spdk application on your ubuntu. (Thank you very much for port on orangepi5plus!)

[Joshua-Riek]

Hmmm, I'm not very familiar with vfio-pci, I will need to do some research.

But have you confirmed that the device "0004:41:00.0" exists? I assume this can be confirmed by the output of lspci.

[rnovikov]

MODINFO name: vfio_pci filename: (builtin) description: VFIO PCI - User Level meta-driver author: Alex Williamson alex.williamson@redhat.com license: GPL v2 file: drivers/vfio/pci/vfio-pci parm: ids:Initial PCI IDs to add to the vfio driver, format is "vendor:device[:subvendor[:subdevice[:class[:class_mask]]]]" and multiple comma separated entries can be specified (string) parm: nointxmask:Disable support for PCI 2.3 style INTx masking. If this resolves problems for specific devices, report lspci -vvvxxx to linux-pci@vger.kernel.org so the device can be fixed automatically via the broken_intx_masking flag. (bool) parm: disable_idle_d3:Disable using the PCI D3 low power state for idle, unused devices (bool) parm: disable_denylist:Disable use of device denylist. Disabling the denylist allows binding to devices with known errata that may lead to exploitable stability or security issues when accessed by untrusted users. (bool)

BINDING PROCESS UNBIND from driver cd /sys/bus/pci/drivers/r8169 ls 0003:31:00.0 0004:41:00.0 bind module new_id remove_id uevent unbind echo 0004:41:00.0 > unbind ls 0003:31:00.0 bind module new_id remove_id uevent unbind cd ../vfio-pci/ ls bind module new_id remove_id uevent unbind echo 0004:41:00.0 > bind echo: write error: No such device dpdk-devbind.py out Other Network devices 0004:41:00.0 'RTL8125 2.5GbE Controller 8125' unused=r8169,vfio-pci

DMESG BOOT ERRORS (if it is needed)

9.869775] Goodix-TS 7-0014: supply AVDD28 not found, using dummy regulator [ 9.871393] Goodix-TS 7-0014: Looking up VDDIO-supply from device tree [ 9.871401] Goodix-TS 7-0014: Looking up VDDIO-supply property in node /i2c@fec90000/touchscreen@14 failed [ 9.871434] Goodix-TS 7-0014: supply VDDIO not found, using dummy regulator [ 9.891880] pwm-fan pwm-fan: Looking up fan-supply from device tree [ 9.891887] pwm-fan pwm-fan: Looking up fan-supply property in node /pwm-fan failed [ 9.964599] Goodix-TS 7-0014: Error reading 1 bytes from 0x8140: -6 [ 9.991643] Goodix-TS 7-0014: Error reading 1 bytes from 0x8140: -6 [ 10.016224] [BT_RFKILL]: bt shut off power [ 10.017912] Goodix-TS 7-0014: I2C communication failure: -6 [ 11.347913] RTL8226B_RTL8221B 2.5Gbps PHY r8169-3-3100:00: attached PHY driver (mii_bus:phy_addr=r8169-3-3100:00, irq=MAC) [ 11.538013] r8169 0003:31:00.0 enP3p49s0: Link is Down [ 11.554575] rk_pcie_establish_link: 228 callbacks suppressed [ 11.554583] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.581246] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.607843] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.634506] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.661173] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.687839] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.714506] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.741173] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.767836] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 11.794506] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3 [ 13.261172] rk-pcie fe170000.pcie: PCIe Link Fail, LTSSM is 0x3, hw_retries=1 [ 14.324508] rk-pcie fe170000.pcie: failed to initialize host

[rnovikov]

@Joshua-Riek If you have time - we can have a debug meeting in google meet with share screen or i can give you a ssh access directly to device. (I am not expecting that we will fixed it, but i am sure we will have fun )

[Joshua-Riek]

From my understanding you need to unbind the pcie network interface first as it's currently in use by the system.

[rnovikov]

in "BINDING PROCESS", as i first step cd /sys/bus/pci/drivers/r8169 ls 0003:31:00.0 0004:41:00.0 bind module new_id remove_id uevent unbind echo 0004:41:00.0 > unbind After this i can sucsesifull bind it back to r8169, but cannot bind to vifo

root@ubuntu:/sys/bus/pci/drivers# echo 0004:41:00.0 > vfio-pci/bind no work bash: echo: write error: No such device root@ubuntu:/sys/bus/pci/drivers# echo 0004:41:00.0 > r8169/bind work

[rnovikov]

I have progress with this issue. cat config-6.1.0-1012-rockchip | grep IOM CONFIG_VIRTIO_IOMMU is not set CONFIG_ROCKCHIP_IOMUX is not set could you give me manual, how to make iso image locally

[Joshua-Riek]

I can enable this in the kernel and push an update tonight. There are a few fixes I already have staged.

[rnovikov]

Thank you very much!

[Joshua-Riek]

Sorry for the delay, I'm waiting on the kernel to build on Launchpad, hopefully it will be done in a few hours.

https://launchpad.net/~jjriek/+archive/ubuntu/rockchip/+build/28434319

[Joshua-Riek]

The kernel upgrade is now ready, any luck using the VFIO drivers?

[rnovikov]

Sorry, long weekend. No access to device till tomorrow.

[rnovikov]

Sorry for delay. Still no work
root@ubuntu:/home/ubuntu# echo 0004:41:00.0 > /sys/bus/pci/drivers/r8169/unbind root@ubuntu:/home/ubuntu# echo 0004:41:00.0 > /sys/bus/pci/devices/0004:41:00.0/driver_override root@ubuntu:/home/ubuntu# echo vfio-pci > /sys/bus/pci/devices/0004:41:00.0/driver_override root@ubuntu:/home/ubuntu# echo 0004:41:00.0 > /sys/bus/pci/drivers/vfio-pci/bind bash: echo: write error: Invalid argument root@ubuntu:/home/ubuntu# echo 0004:41:00.0 > /sys/bus/pci/drivers/vfio-pci/bind bash: echo: write error: Invalid argument

[rnovikov]

I dig a bit dipper. Same problem on ubuntu 22.04. with NIC (0004:41:00.0) and with SSD (0000:01:00.0)