Support assigning Thunderbolt ports to VMs

[DemiMarie]

Qubes OS version:

R4.0

Affected component(s):

dom0 kernel, qubes-core-admin

---

Steps to reproduce the behavior:

Try to assign a Thunderbolt port to a VM. This is incredibly useful for e.g. external graphics cards, which (to my understanding) are the only way to do gaming/GPGPU in Qubes without a security risk.

Expected behavior:

Thunderbolt can be passed through in the usual way.

Actual behavior:

No way to figure out which PCI device is the Thunderbolt device.

General notes:

My understanding is that this makes doing GPGPU or gaming on Qubes in a supported and secure configuration impossible.

---

Related issues:

[andrewdavidwong]

Related issue: #2454

[DemiMarie]

I think I figured out the problem:

Because our kernel is built with CONFIG_PCI_HOTPLUG=N, it simply doesn’t recognize the PCI devices. At all.

Instead, we need something similar to our situation with USB devices, where the kernel recognizes the device, but doesn’t attempt to do anything with them (assigning the pciback driver to them). That allows the devices to then be assigned to a VM.

[marmarek]

Does it also happen if you boot the system with the device plugged in? If it doesn't work then either, the problem is somewhere else. Anyway, I think just having CONFIG_PCI_HOTPLUG enabled isn't enough for hotplug to the VM work. Something needs to also (re-)configure IOMMU when you connect the device. AFAIK this isn't handled automatically right now.

As for "doesn’t attempt to do anything with them", Thunderbolt supports some kind of device connection policy. It should be possible to use that to prevent device to access any resources (especially DMA to system RAM) before it gets accepted (for example assigned to a VM. But I don't know details, including support for that in Linux and Xen. Without thing like this, enabling PCI hotplug is risky, because PCI devices generally (unless properly isolated using IOMMU) can access system RAM using DMA.

[DemiMarie]

I am not talking about assigning the individual Thunderbolt device, but rather the entire controller. Does that make sense?

On Sep 29, 2018 5:10 PM, "Marek Marczykowski-Górecki" < notifications@github.com> wrote:

Does it also happen if you boot the system with the device plugged in? If it doesn't work then either, the problem is somewhere else. Anyway, I think just having CONFIG_PCI_HOTPLUG enabled isn't enough for hotplug to the VM work. Something needs to also (re-)configure IOMMU when you connect the device. AFAIK this isn't handled automatically right now.

As for "doesn’t attempt to do anything with them", Thunderbolt supports some kind of device connection policy. It should be possible to use that to prevent device to access any resources (especially DMA to system RAM) before it gets accepted (for example assigned to a VM. But I don't know details, including support for that in Linux and Xen. Without thing like this, enabling PCI hotplug is risky, because PCI devices generally (unless properly isolated using IOMMU) can access system RAM using DMA.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/QubesOS/qubes-issues/issues/4353#issuecomment-425675791, or mute the thread https://github.com/notifications/unsubscribe-auth/AGGWB9JGm83OdncbiDfqORqAQrCTuaRhks5uf-HcgaJpZM4W-3gI .

[marmarek]

Connected devices are separate PCI devices. Behind a bridge (separate for each port), but still a device, which needs to be properly guarded by IOMMU. Assigning just the bridge (which to be honest I'm not sure if Xen supports at all) isn't enough.

Anyway, you can try, lspci and lspci -t output in dom0 should be helpful in figuring out which device is connected where.

[DemiMarie]

This will be fixed when #6620 is. Xen has safe support for PCIe hotplug as of XSA-306.

[swenew]

Hello, has anyone found a solution to enable Thunderbold 3?

[DemiMarie]

This one is up to @marmarek. I am not sure how to test Xen’s IOMMU handling.