search

mcusim / freebsd-src

sys/dev/dpaa2 drivers work-in-progress
https://www.FreeBSD.org/
Other
4 stars 3 forks source link

Support working under VFIO passthrough #9

Closed mcbridematt closed 2 years ago

mcbridematt commented 2 years ago

This fixes some issues that were preventing the driver from working under VFIO / QEMU passthrough. This has been tested with muvirt on Ten64 and MC firmware 10.20 (see our forum on how to set this up: https://forum.traverse.com.au/t/restool-in-muvirt/63/17?u=mcbridematt).

There are some changes to VFIO in more recent MC firmwares (>10.24) and QEMU versions so some adjustments may still be required.

Key changes:

  1. Extend ICID to 32-bits. The top 16 bits (marked as 'reserved' in NXP documentation) are used to differentiate host and guest interrupts in the (v)GIC [interrupt controller]. Without the full ICID the interrupts will not be routed in the kernel 
    dpaa2_rc0: Isolation context ID: 23 # Bare metal
dpaa2_rc0: Isolation context ID: 65536 # VM
  2. Treat an unreachable DPMAC (when we ask if it is PHY or FIXED) as a FIXED link. (This is because the DPMAC is usually sitting in the host / root DPRC and cannot be accessed by the child DPRC) Ideally, this would be avoided by having the DPMAC driver 'own' the PHY connection (signalling the MC on MII/MDIO events), and the DPNI receiving it's link state indications from the MC via interrupts. However, it's not worth the effort to implement this unless FreeBSD gains the ability to act as a host for VFIO passthrough.
  3. Support non-DPMAC partner types: DPNI, DPDMUX, DPSW. These function exactly the same as a DPMAC in FIXED mode, but their MAC addresses are set in the DPNI object only, so move the MAC query out of the DPMAC only section.
dsalychev commented 2 years ago

Patch applied starting from ef64c6e0edca3a68aede5c796ca87d35bcab49bb. Thanks!