search

pavolelsig / passthrough_helper_manjaro

67 stars 18 forks source link

passing primary x16 slot #9

Open dibg opened 3 years ago

dibg commented 3 years ago

First i must say you did a great work i found the videos and this scrip to be a great help and i did manage to pass the p2000 quadro when its on the second slot. Due to the bandwidth limitation (x4pcie3.0) of the second slot i have hard time using looking glass so i decided to switch the gpus places. 2 days later i didn't managed to make this to fucntion.

Here is some usefull information: my motherboard is msi x570 mpg gaming plus. I has connected only the first slot to the gpu and the other one thought the chipset. It dosen't have option to choose what gpu to boot with. I do use bios for booting my linux not uefi. The gpu i try to pass is a quadro p2000 so i don't have code 43 when i have it on the second slot but i have when im passing it from the fisr slot. My linux gpu is gtx 1050ti.

I did modify the script https://github.com/pavolelsig/passthrough_helper_manjaro/blob/master/vfio-pci-override.sh override only the quadro

!/bin/sh

echo "vfio-pci" > /sys/bus/pci/devices/0000:2d:00.0/driver_override echo "vfio-pci" > /sys/bus/pci/devices/0000:2d:00.1/driver_override modprobe -i vfio-pci

And i managed to configure the xord to user the gtx1050ti to display.

During booting the quadro in the top slot (and therefore primary output device) shows signal until the vfio hook message. in the vm i always get code 43 from nvidia. ( i used bios in the windows vm) I thougth maybe i can hide the kvm from the windows but it din't help. I did pull the vbios out of the card from a windows bare metal installation and provided into the vm and it does show image if i switch into the uefi until the actual windows booting starting where it freezes show the uefi booting screen(picture below)

what can i do to pass the gpu from the first slot ?

00:00.0 0600: 1022:1480
    Subsystem: 1462:7c37
    Flags: fast devsel

00:00.2 0806: 1022:1481
    Subsystem: 1022:1481
    Flags: fast devsel, IRQ 26
    Capabilities: [40] Secure device <?>
    Capabilities: [64] MSI: Enable+ Count=1/4 Maskable- 64bit+
    Capabilities: [74] HyperTransport: MSI Mapping Enable+ Fixed+
    Capabilities: [c8] Vendor Specific Information: Len=18 <?>

00:01.0 0600: 1022:1482
    Flags: fast devsel, IOMMU group 0

00:01.1 0604: 1022:1483 (prog-if 00 [Normal decode])
    Flags: bus master, fast devsel, latency 0, IRQ 27, IOMMU group 1
    Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
    I/O behind bridge: [disabled]
    Memory behind bridge: fc400000-fc4fffff [size=1M]
    Prefetchable memory behind bridge: [disabled]
    Capabilities: [50] Power Management version 3
    Capabilities: [58] Express Root Port (Slot+), MSI 00
    Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Capabilities: [c0] Subsystem: 1022:1453
    Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [150] Advanced Error Reporting
    Capabilities: [270] Secondary PCI Express
    Capabilities: [2a0] Access Control Services
    Capabilities: [370] L1 PM Substates
    Capabilities: [3c4] Designated Vendor-Specific: Vendor=1022 ID=0001 Rev=1 Len=44 <?>
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>
    Kernel driver in use: pcieport

00:01.2 0604: 1022:1483 (prog-if 00 [Normal decode])
    Flags: bus master, fast devsel, latency 0, IRQ 28, IOMMU group 2
    Bus: primary=00, secondary=20, subordinate=2c, sec-latency=0
    I/O behind bridge: 0000c000-0000efff [size=12K]
    Memory behind bridge: f9000000-fa9fffff [size=26M]
    Prefetchable memory behind bridge: 00000000d0000000-00000000e1ffffff [size=288M]
    Capabilities: [50] Power Management version 3
    Capabilities: [58] Express Root Port (Slot+), MSI 00
    Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Capabilities: [c0] Subsystem: 1022:1453
    Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [150] Advanced Error Reporting
    Capabilities: [270] Secondary PCI Express
    Capabilities: [2a0] Access Control Services
    Capabilities: [370] L1 PM Substates
    Capabilities: [3c4] Designated Vendor-Specific: Vendor=1022 ID=0001 Rev=1 Len=44 <?>
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>
    Kernel driver in use: pcieport

00:02.0 0600: 1022:1482
    Flags: fast devsel, IOMMU group 3

00:03.0 0600: 1022:1482
    Flags: fast devsel, IOMMU group 4

00:03.1 0604: 1022:1483 (prog-if 00 [Normal decode])
    Flags: bus master, fast devsel, latency 0, IRQ 29, IOMMU group 5
    Bus: primary=00, secondary=2d, subordinate=2d, sec-latency=0
    I/O behind bridge: 0000f000-0000ffff [size=4K]
    Memory behind bridge: fb000000-fc0fffff [size=17M]
    Prefetchable memory behind bridge: 00000000b0000000-00000000c1ffffff [size=288M]
    Capabilities: [50] Power Management version 3
    Capabilities: [58] Express Root Port (Slot+), MSI 00
    Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Capabilities: [c0] Subsystem: 1022:1453
    Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [150] Advanced Error Reporting
    Capabilities: [270] Secondary PCI Express
    Capabilities: [2a0] Access Control Services
    Capabilities: [370] L1 PM Substates
    Capabilities: [3c4] Designated Vendor-Specific: Vendor=1022 ID=0001 Rev=1 Len=44 <?>
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>
    Kernel driver in use: pcieport

00:04.0 0600: 1022:1482
    Flags: fast devsel, IOMMU group 6

00:05.0 0600: 1022:1482
    Flags: fast devsel, IOMMU group 7

00:07.0 0600: 1022:1482
    Flags: fast devsel, IOMMU group 8

00:07.1 0604: 1022:1484 (prog-if 00 [Normal decode])
    Flags: bus master, fast devsel, latency 0, IRQ 31, IOMMU group 9
    Bus: primary=00, secondary=2e, subordinate=2e, sec-latency=0
    I/O behind bridge: [disabled]
    Memory behind bridge: [disabled]
    Prefetchable memory behind bridge: [disabled]
    Capabilities: [50] Power Management version 3
    Capabilities: [58] Express Root Port (Slot-), MSI 00
    Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Capabilities: [c0] Subsystem: 1022:1484
    Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [150] Advanced Error Reporting
    Capabilities: [270] Secondary PCI Express
    Capabilities: [2a0] Access Control Services
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>
    Kernel driver in use: pcieport

00:08.0 0600: 1022:1482
    Flags: fast devsel, IOMMU group 10

00:08.1 0604: 1022:1484 (prog-if 00 [Normal decode])
    Flags: bus master, fast devsel, latency 0, IRQ 32, IOMMU group 11
    Bus: primary=00, secondary=2f, subordinate=2f, sec-latency=0
    I/O behind bridge: [disabled]
    Memory behind bridge: fc200000-fc3fffff [size=2M]
    Prefetchable memory behind bridge: [disabled]
    Capabilities: [50] Power Management version 3
    Capabilities: [58] Express Root Port (Slot-), MSI 00
    Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Capabilities: [c0] Subsystem: 1022:1484
    Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [150] Advanced Error Reporting
    Capabilities: [270] Secondary PCI Express
    Capabilities: [2a0] Access Control Services
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>
    Kernel driver in use: pcieport

00:14.0 0c05: 1022:790b (rev 61)
    Subsystem: 1462:7c37
    Flags: 66MHz, medium devsel, IOMMU group 12
    Kernel driver in use: piix4_smbus
    Kernel modules: i2c_piix4, sp5100_tco

00:14.3 0601: 1022:790e (rev 51)
    Subsystem: 1462:7c37
    Flags: bus master, 66MHz, medium devsel, latency 0, IOMMU group 12

00:18.0 0600: 1022:1440
    Flags: fast devsel, IOMMU group 13

00:18.1 0600: 1022:1441
    Flags: fast devsel, IOMMU group 13

00:18.2 0600: 1022:1442
    Flags: fast devsel, IOMMU group 13

00:18.3 0600: 1022:1443
    Flags: fast devsel, IOMMU group 13
    Kernel driver in use: zenpower
    Kernel modules: k10temp, zenpower

00:18.4 0600: 1022:1444
    Flags: fast devsel, IOMMU group 13

00:18.5 0600: 1022:1445
    Flags: fast devsel, IOMMU group 13

00:18.6 0600: 1022:1446
    Flags: fast devsel, IOMMU group 13

00:18.7 0600: 1022:1447
    Flags: fast devsel, IOMMU group 13

01:00.0 0108: 1987:5012 (rev 01) (prog-if 02 [NVM Express])
    Subsystem: 1987:5012
    Flags: bus master, fast devsel, latency 0, IRQ 43, NUMA node 0, IOMMU group 14
    Memory at fc400000 (64-bit, non-prefetchable) [size=16K]
    Capabilities: [80] Express Endpoint, MSI 00
    Capabilities: [d0] MSI-X: Enable+ Count=9 Masked-
    Capabilities: [e0] MSI: Enable- Count=1/8 Maskable- 64bit+
    Capabilities: [f8] Power Management version 3
    Capabilities: [100] Latency Tolerance Reporting
    Capabilities: [110] L1 PM Substates
    Capabilities: [128] Alternative Routing-ID Interpretation (ARI)
    Capabilities: [200] Advanced Error Reporting
    Capabilities: [300] Secondary PCI Express
    Kernel driver in use: nvme

20:00.0 0604: 1022:57ad (prog-if 00 [Normal decode])
    Flags: bus master, fast devsel, latency 0, IRQ 24, IOMMU group 15
    Bus: primary=20, secondary=21, subordinate=2c, sec-latency=0
    I/O behind bridge: 0000c000-0000efff [size=12K]
    Memory behind bridge: f9000000-fa9fffff [size=26M]
    Prefetchable memory behind bridge: 00000000d0000000-00000000e1ffffff [size=288M]
    Capabilities: [50] Power Management version 3
    Capabilities: [58] Express Upstream Port, MSI 00
    Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [270] Secondary PCI Express
    Capabilities: [370] L1 PM Substates
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>
    Kernel driver in use: pcieport

21:00.0 0604: 1022:57a3 (prog-if 00 [Normal decode])
    Flags: bus master, fast devsel, latency 0, IRQ 33, IOMMU group 16
    Bus: primary=21, secondary=22, subordinate=22, sec-latency=0
    I/O behind bridge: [disabled]
    Memory behind bridge: fa900000-fa9fffff [size=1M]
    Prefetchable memory behind bridge: [disabled]
    Capabilities: [50] Power Management version 3
    Capabilities: [58] Express Downstream Port (Slot+), MSI 00
    Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Capabilities: [c0] Subsystem: 1022:1453
    Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [150] Advanced Error Reporting
    Capabilities: [270] Secondary PCI Express
    Capabilities: [2a0] Access Control Services
    Capabilities: [370] L1 PM Substates
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>
    Kernel driver in use: pcieport

21:01.0 0604: 1022:57a3 (prog-if 00 [Normal decode])
    Flags: bus master, fast devsel, latency 0, IRQ 34, IOMMU group 17
    Bus: primary=21, secondary=23, subordinate=23, sec-latency=0
    I/O behind bridge: 0000e000-0000efff [size=4K]
    Memory behind bridge: f9000000-fa0fffff [size=17M]
    Prefetchable memory behind bridge: 00000000d0000000-00000000e1ffffff [size=288M]
    Capabilities: [50] Power Management version 3
    Capabilities: [58] Express Downstream Port (Slot+), MSI 00
    Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Capabilities: [c0] Subsystem: 1022:1453
    Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [150] Advanced Error Reporting
    Capabilities: [270] Secondary PCI Express
    Capabilities: [2a0] Access Control Services
    Capabilities: [370] L1 PM Substates
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>
    Kernel driver in use: pcieport

21:04.0 0604: 1022:57a3 (prog-if 00 [Normal decode])
    Flags: bus master, fast devsel, latency 0, IRQ 35, IOMMU group 18
    Bus: primary=21, secondary=26, subordinate=26, sec-latency=0
    I/O behind bridge: 0000d000-0000dfff [size=4K]
    Memory behind bridge: fa800000-fa8fffff [size=1M]
    Prefetchable memory behind bridge: [disabled]
    Capabilities: [50] Power Management version 3
    Capabilities: [58] Express Downstream Port (Slot+), MSI 00
    Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Capabilities: [c0] Subsystem: 1022:1453
    Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [150] Advanced Error Reporting
    Capabilities: [270] Secondary PCI Express
    Capabilities: [2a0] Access Control Services
    Capabilities: [370] L1 PM Substates
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>
    Kernel driver in use: pcieport

21:05.0 0604: 1022:57a3 (prog-if 00 [Normal decode])
    Flags: bus master, fast devsel, latency 0, IRQ 36, IOMMU group 19
    Bus: primary=21, secondary=27, subordinate=27, sec-latency=0
    I/O behind bridge: 0000c000-0000cfff [size=4K]
    Memory behind bridge: fa700000-fa7fffff [size=1M]
    Prefetchable memory behind bridge: [disabled]
    Capabilities: [50] Power Management version 3
    Capabilities: [58] Express Downstream Port (Slot+), MSI 00
    Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Capabilities: [c0] Subsystem: 1022:1453
    Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [150] Advanced Error Reporting
    Capabilities: [270] Secondary PCI Express
    Capabilities: [2a0] Access Control Services
    Capabilities: [370] L1 PM Substates
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>
    Kernel driver in use: pcieport

21:06.0 0604: 1022:57a3 (prog-if 00 [Normal decode])
    Flags: bus master, fast devsel, latency 0, IRQ 37, IOMMU group 20
    Bus: primary=21, secondary=28, subordinate=28, sec-latency=0
    I/O behind bridge: [disabled]
    Memory behind bridge: fa600000-fa6fffff [size=1M]
    Prefetchable memory behind bridge: [disabled]
    Capabilities: [50] Power Management version 3
    Capabilities: [58] Express Downstream Port (Slot+), MSI 00
    Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Capabilities: [c0] Subsystem: 1022:1453
    Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [150] Advanced Error Reporting
    Capabilities: [270] Secondary PCI Express
    Capabilities: [2a0] Access Control Services
    Capabilities: [370] L1 PM Substates
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>
    Kernel driver in use: pcieport

21:08.0 0604: 1022:57a4 (prog-if 00 [Normal decode])
    Flags: bus master, fast devsel, latency 0, IRQ 38, IOMMU group 21
    Bus: primary=21, secondary=2a, subordinate=2a, sec-latency=0
    I/O behind bridge: [disabled]
    Memory behind bridge: fa200000-fa3fffff [size=2M]
    Prefetchable memory behind bridge: [disabled]
    Capabilities: [50] Power Management version 3
    Capabilities: [58] Express Downstream Port (Slot-), MSI 00
    Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Capabilities: [c0] Subsystem: 1022:1484
    Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [270] Secondary PCI Express
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>
    Kernel driver in use: pcieport

21:09.0 0604: 1022:57a4 (prog-if 00 [Normal decode])
    Flags: bus master, fast devsel, latency 0, IRQ 40, IOMMU group 22
    Bus: primary=21, secondary=2b, subordinate=2b, sec-latency=0
    I/O behind bridge: [disabled]
    Memory behind bridge: fa500000-fa5fffff [size=1M]
    Prefetchable memory behind bridge: [disabled]
    Capabilities: [50] Power Management version 3
    Capabilities: [58] Express Downstream Port (Slot-), MSI 00
    Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Capabilities: [c0] Subsystem: 1022:1484
    Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [270] Secondary PCI Express
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>
    Kernel driver in use: pcieport

21:0a.0 0604: 1022:57a4 (prog-if 00 [Normal decode])
    Flags: bus master, fast devsel, latency 0, IRQ 42, IOMMU group 23
    Bus: primary=21, secondary=2c, subordinate=2c, sec-latency=0
    I/O behind bridge: [disabled]
    Memory behind bridge: fa400000-fa4fffff [size=1M]
    Prefetchable memory behind bridge: [disabled]
    Capabilities: [50] Power Management version 3
    Capabilities: [58] Express Downstream Port (Slot-), MSI 00
    Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Capabilities: [c0] Subsystem: 1022:1484
    Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [270] Secondary PCI Express
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>
    Kernel driver in use: pcieport

22:00.0 0108: 15b7:5009 (rev 01) (prog-if 02 [NVM Express])
    Subsystem: 15b7:5009
    Flags: bus master, fast devsel, latency 0, IRQ 24, NUMA node 0, IOMMU group 24
    Memory at fa900000 (64-bit, non-prefetchable) [size=16K]
    Memory at fa904000 (64-bit, non-prefetchable) [size=256]
    Capabilities: [80] Power Management version 3
    Capabilities: [90] MSI: Enable- Count=1/32 Maskable- 64bit+
    Capabilities: [b0] MSI-X: Enable+ Count=17 Masked-
    Capabilities: [c0] Express Endpoint, MSI 00
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [150] Device Serial Number 00-00-00-00-00-00-00-00
    Capabilities: [1b8] Latency Tolerance Reporting
    Capabilities: [300] Secondary PCI Express
    Capabilities: [900] L1 PM Substates
    Kernel driver in use: nvme

23:00.0 0300: 10de:1c82 (rev a1) (prog-if 00 [VGA controller])
    Subsystem: 1458:378e
    Flags: bus master, fast devsel, latency 0, IRQ 126, IOMMU group 25
    Memory at f9000000 (32-bit, non-prefetchable) [size=16M]
    Memory at d0000000 (64-bit, prefetchable) [size=256M]
    Memory at e0000000 (64-bit, prefetchable) [size=32M]
    I/O ports at e000 [size=128]
    Expansion ROM at fa000000 [virtual] [disabled] [size=512K]
    Capabilities: [60] Power Management version 3
    Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Capabilities: [78] Express Legacy Endpoint, MSI 00
    Capabilities: [100] Virtual Channel
    Capabilities: [250] Latency Tolerance Reporting
    Capabilities: [128] Power Budgeting <?>
    Capabilities: [420] Advanced Error Reporting
    Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
    Capabilities: [900] Secondary PCI Express
    Kernel driver in use: nvidia
    Kernel modules: nouveau, nvidia_drm, nvidia

23:00.1 0403: 10de:0fb9 (rev a1)
    Subsystem: 1458:378e
    Flags: bus master, fast devsel, latency 0, IRQ 41, IOMMU group 25
    Memory at fa080000 (32-bit, non-prefetchable) [size=16K]
    Capabilities: [60] Power Management version 3
    Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
    Capabilities: [78] Express Endpoint, MSI 00
    Capabilities: [100] Advanced Error Reporting
    Kernel driver in use: snd_hda_intel
    Kernel modules: snd_hda_intel

26:00.0 0106: 1b21:0612 (rev 02) (prog-if 01 [AHCI 1.0])
    Subsystem: 1b21:1060
    Flags: bus master, fast devsel, latency 0, IRQ 105, IOMMU group 26
    I/O ports at d050 [size=8]
    I/O ports at d040 [size=4]
    I/O ports at d030 [size=8]
    I/O ports at d020 [size=4]
    I/O ports at d000 [size=32]
    Memory at fa800000 (32-bit, non-prefetchable) [size=512]
    Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit-
    Capabilities: [78] Power Management version 3
    Capabilities: [80] Express Legacy Endpoint, MSI 00
    Capabilities: [100] Virtual Channel
    Kernel driver in use: ahci
    Kernel modules: ahci

27:00.0 0200: 10ec:8168 (rev 15)
    Subsystem: 1462:7c37
    Flags: bus master, fast devsel, latency 0, IRQ 39, IOMMU group 27
    I/O ports at c000 [size=256]
    Memory at fa704000 (64-bit, non-prefetchable) [size=4K]
    Memory at fa700000 (64-bit, non-prefetchable) [size=16K]
    Capabilities: [40] Power Management version 3
    Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
    Capabilities: [70] Express Endpoint, MSI 01
    Capabilities: [b0] MSI-X: Enable+ Count=4 Masked-
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [140] Virtual Channel
    Capabilities: [160] Device Serial Number 01-00-00-00-68-4c-e0-00
    Capabilities: [170] Latency Tolerance Reporting
    Capabilities: [178] L1 PM Substates
    Kernel driver in use: r8169
    Kernel modules: r8169

28:00.0 0108: 15b7:5009 (rev 01) (prog-if 02 [NVM Express])
    DeviceName: RTL8111EPV
    Subsystem: 15b7:5009
    Flags: bus master, fast devsel, latency 0, IRQ 41, NUMA node 0, IOMMU group 28
    Memory at fa600000 (64-bit, non-prefetchable) [size=16K]
    Memory at fa604000 (64-bit, non-prefetchable) [size=256]
    Capabilities: [80] Power Management version 3
    Capabilities: [90] MSI: Enable- Count=1/32 Maskable- 64bit+
    Capabilities: [b0] MSI-X: Enable+ Count=17 Masked-
    Capabilities: [c0] Express Endpoint, MSI 00
    Capabilities: [100] Advanced Error Reporting
    Capabilities: [150] Device Serial Number 00-00-00-00-00-00-00-00
    Capabilities: [1b8] Latency Tolerance Reporting
    Capabilities: [300] Secondary PCI Express
    Capabilities: [900] L1 PM Substates
    Kernel driver in use: nvme

2a:00.0 1300: 1022:1485
    Subsystem: 1022:1485
    Flags: bus master, fast devsel, latency 0, IOMMU group 21
    Capabilities: [48] Vendor Specific Information: Len=08 <?>
    Capabilities: [50] Power Management version 3
    Capabilities: [64] Express Endpoint, MSI 00
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [270] Secondary PCI Express
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>

2a:00.1 0c03: 1022:149c (prog-if 30 [XHCI])
    Subsystem: 1022:1486
    Flags: bus master, fast devsel, latency 0, IRQ 87, IOMMU group 21
    Memory at fa300000 (64-bit, non-prefetchable) [size=1M]
    Capabilities: [48] Vendor Specific Information: Len=08 <?>
    Capabilities: [50] Power Management version 3
    Capabilities: [64] Express Endpoint, MSI 00
    Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Kernel driver in use: xhci_hcd
    Kernel modules: xhci_pci

2a:00.3 0c03: 1022:149c (prog-if 30 [XHCI])
    Subsystem: 1022:148c
    Flags: bus master, fast devsel, latency 0, IRQ 41, IOMMU group 21
    Memory at fa200000 (64-bit, non-prefetchable) [size=1M]
    Capabilities: [48] Vendor Specific Information: Len=08 <?>
    Capabilities: [50] Power Management version 3
    Capabilities: [64] Express Endpoint, MSI 00
    Capabilities: [a0] MSI: Enable- Count=1/8 Maskable- 64bit+
    Capabilities: [c0] MSI-X: Enable+ Count=8 Masked-
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Kernel driver in use: xhci_hcd
    Kernel modules: xhci_pci

2b:00.0 0106: 1022:7901 (rev 51) (prog-if 01 [AHCI 1.0])
    Subsystem: 1022:7901
    Flags: bus master, fast devsel, latency 0, IRQ 106, IOMMU group 22
    Memory at fa500000 (32-bit, non-prefetchable) [size=2K]
    Capabilities: [48] Vendor Specific Information: Len=08 <?>
    Capabilities: [50] Power Management version 3
    Capabilities: [64] Express Endpoint, MSI 00
    Capabilities: [a0] MSI: Enable+ Count=1/16 Maskable- 64bit+
    Capabilities: [d0] SATA HBA v1.0
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [270] Secondary PCI Express
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>
    Kernel driver in use: ahci
    Kernel modules: ahci

2c:00.0 0106: 1022:7901 (rev 51) (prog-if 01 [AHCI 1.0])
    Subsystem: 1022:7901
    Flags: bus master, fast devsel, latency 0, IRQ 107, IOMMU group 23
    Memory at fa400000 (32-bit, non-prefetchable) [size=2K]
    Capabilities: [48] Vendor Specific Information: Len=08 <?>
    Capabilities: [50] Power Management version 3
    Capabilities: [64] Express Endpoint, MSI 00
    Capabilities: [a0] MSI: Enable+ Count=16/16 Maskable- 64bit+
    Capabilities: [d0] SATA HBA v1.0
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [270] Secondary PCI Express
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>
    Kernel driver in use: ahci
    Kernel modules: ahci

2d:00.0 0300: 10de:1c30 (rev a1) (prog-if 00 [VGA controller])
    Subsystem: 103c:11b3
    Flags: fast devsel, IRQ 49, IOMMU group 29
    Memory at fb000000 (32-bit, non-prefetchable) [size=16M]
    Memory at b0000000 (64-bit, prefetchable) [size=256M]
    Memory at c0000000 (64-bit, prefetchable) [size=32M]
    I/O ports at f000 [size=128]
    Expansion ROM at 000c0000 [disabled] [size=128K]
    Capabilities: [60] Power Management version 3
    Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
    Capabilities: [78] Express Legacy Endpoint, MSI 00
    Capabilities: [100] Virtual Channel
    Capabilities: [250] Latency Tolerance Reporting
    Capabilities: [128] Power Budgeting <?>
    Capabilities: [420] Advanced Error Reporting
    Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
    Capabilities: [900] Secondary PCI Express
    Kernel driver in use: vfio-pci
    Kernel modules: nouveau, nvidia_drm, nvidia

2d:00.1 0403: 10de:10f1 (rev a1)
    Subsystem: 103c:11b3
    Flags: fast devsel, IRQ 51, IOMMU group 29
    Memory at fc080000 (32-bit, non-prefetchable) [size=16K]
    Capabilities: [60] Power Management version 3
    Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
    Capabilities: [78] Express Endpoint, MSI 00
    Capabilities: [100] Advanced Error Reporting
    Kernel driver in use: vfio-pci
    Kernel modules: snd_hda_intel

2e:00.0 1300: 1022:148a
    Subsystem: 1022:148a
    Flags: bus master, fast devsel, latency 0, IOMMU group 30
    Capabilities: [48] Vendor Specific Information: Len=08 <?>
    Capabilities: [50] Power Management version 3
    Capabilities: [64] Express Endpoint, MSI 00
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [150] Advanced Error Reporting
    Capabilities: [270] Secondary PCI Express
    Capabilities: [2a0] Access Control Services
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>

2f:00.0 1300: 1022:1485
    Subsystem: 1022:1485
    Flags: bus master, fast devsel, latency 0, IOMMU group 31
    Capabilities: [48] Vendor Specific Information: Len=08 <?>
    Capabilities: [50] Power Management version 3
    Capabilities: [64] Express Endpoint, MSI 00
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [150] Advanced Error Reporting
    Capabilities: [270] Secondary PCI Express
    Capabilities: [2a0] Access Control Services
    Capabilities: [400] Data Link Feature <?>
    Capabilities: [410] Physical Layer 16.0 GT/s <?>
    Capabilities: [440] Lane Margining at the Receiver <?>

2f:00.3 0c03: 1022:149c (prog-if 30 [XHCI])
    Subsystem: 1462:7c37
    Flags: bus master, fast devsel, latency 0, IRQ 96, IOMMU group 32
    Memory at fc200000 (64-bit, non-prefetchable) [size=1M]
    Capabilities: [48] Vendor Specific Information: Len=08 <?>
    Capabilities: [50] Power Management version 3
    Capabilities: [64] Express Endpoint, MSI 00
    Capabilities: [a0] MSI: Enable- Count=1/8 Maskable- 64bit+
    Capabilities: [c0] MSI-X: Enable+ Count=8 Masked-
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [150] Advanced Error Reporting
    Capabilities: [2a0] Access Control Services
    Capabilities: [370] Transaction Processing Hints
    Kernel driver in use: xhci_hcd
    Kernel modules: xhci_pci

2f:00.4 0403: 1022:1487
    Subsystem: 1462:ec37
    Flags: bus master, fast devsel, latency 0, IRQ 44, IOMMU group 33
    Memory at fc300000 (32-bit, non-prefetchable) [size=32K]
    Capabilities: [48] Vendor Specific Information: Len=08 <?>
    Capabilities: [50] Power Management version 3
    Capabilities: [64] Express Endpoint, MSI 00
    Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
    Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [150] Advanced Error Reporting
    Capabilities: [2a0] Access Control Services
    Capabilities: [370] Transaction Processing Hints
    Kernel driver in use: snd_hda_intel
    Kernel modules: snd_hda_intel
<domain type="kvm">
  <name>win10</name>
  <uuid>7f83a89c-f715-40cf-b654-63bd6c26b344</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://microsoft.com/win/10"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit="KiB">33157120</memory>
  <currentMemory unit="KiB">33157120</currentMemory>
  <memoryBacking>
    <hugepages/>
  </memoryBacking>
  <vcpu placement="static">12</vcpu>
  <cputune>
    <vcpupin vcpu="0" cpuset="18"/>
    <vcpupin vcpu="1" cpuset="9"/>
    <vcpupin vcpu="2" cpuset="20"/>
    <vcpupin vcpu="3" cpuset="10"/>
    <vcpupin vcpu="4" cpuset="22"/>
    <vcpupin vcpu="5" cpuset="11"/>
    <vcpupin vcpu="6" cpuset="12"/>
    <vcpupin vcpu="7" cpuset="6"/>
    <vcpupin vcpu="8" cpuset="14"/>
    <vcpupin vcpu="9" cpuset="7"/>
    <vcpupin vcpu="10" cpuset="16"/>
    <vcpupin vcpu="11" cpuset="8"/>
  </cputune>
  <os>
    <type arch="x86_64" machine="pc-q35-5.2">hvm</type>
    <loader readonly="yes" type="pflash">/usr/share/edk2-ovmf/x64/OVMF_CODE.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/win10_VARS.fd</nvram>
    <bootmenu enable="yes"/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state="on"/>
      <vapic state="on"/>
      <spinlocks state="on" retries="8191"/>
    </hyperv>
    <vmport state="off"/>
  </features>
  <cpu mode="host-model" check="partial">
    <topology sockets="1" dies="1" cores="6" threads="2"/>
  </cpu>
  <clock offset="localtime">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
    <timer name="hypervclock" present="yes"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <controller type="usb" index="0" model="qemu-xhci" ports="15">
      <address type="pci" domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
    </controller>
    <controller type="sata" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1f" function="0x2"/>
    </controller>
    <controller type="pci" index="0" model="pcie-root"/>
    <controller type="pci" index="1" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="1" port="0x10"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="2" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="2" port="0x11"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x1"/>
    </controller>
    <controller type="pci" index="3" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="3" port="0x12"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x2"/>
    </controller>
    <controller type="pci" index="4" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="4" port="0x13"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x3"/>
    </controller>
    <controller type="pci" index="5" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="5" port="0x14"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x4"/>
    </controller>
    <controller type="pci" index="6" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="6" port="0x15"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x5"/>
    </controller>
    <controller type="pci" index="7" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="7" port="0x16"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x6"/>
    </controller>
    <controller type="pci" index="8" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="8" port="0x17"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x7"/>
    </controller>
    <controller type="pci" index="9" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="9" port="0x18"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0"/>
    </controller>
    <controller type="pci" index="10" model="pcie-to-pci-bridge">
      <model name="pcie-pci-bridge"/>
      <address type="pci" domain="0x0000" bus="0x08" slot="0x00" function="0x0"/>
    </controller>
    <controller type="virtio-serial" index="0">
      <address type="pci" domain="0x0000" bus="0x03" slot="0x00" function="0x0"/>
    </controller>
    <interface type="bridge">
      <mac address="52:54:00:05:78:e7"/>
      <source bridge="bridge0"/>
      <model type="e1000e"/>
      <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
    </interface>
    <serial type="pty">
      <target type="isa-serial" port="0">
        <model name="isa-serial"/>
      </target>
    </serial>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
    <channel type="spicevmc">
      <target type="virtio" name="com.redhat.spice.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="2"/>
    </channel>
    <input type="mouse" bus="ps2"/>
    <input type="keyboard" bus="ps2"/>
    <graphics type="spice" autoport="yes">
      <listen type="address"/>
      <image compression="off"/>
    </graphics>
    <sound model="ich9">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1b" function="0x0"/>
    </sound>
    <video>
      <model type="qxl" ram="65536" vram="65536" vgamem="16384" heads="1" primary="yes"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>
    </video>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x28" slot="0x00" function="0x0"/>
      </source>
      <boot order="1"/>
      <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x2d" slot="0x00" function="0x0"/>
      </source>
      <rom file="/home/user/Documents/Dumps/p2000_win_e.rom"/>
      <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x2d" slot="0x00" function="0x1"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/>
    </hostdev>
    <memballoon model="virtio">
      <address type="pci" domain="0x0000" bus="0x07" slot="0x00" function="0x0"/>
    </memballoon>
    <shmem name="looking-glass">
      <model type="ivshmem-plain"/>
      <size unit="M">64</size>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x0b" function="0x0"/>
    </shmem>
    <shmem name="scream-ivshmem">
      <model type="ivshmem-plain"/>
      <size unit="M">2</size>
      <address type="pci" domain="0x0000" bus="0x0a" slot="0x01" function="0x0"/>
    </shmem>
  </devices>
</domain>

image

vfio43

pavolelsig commented 3 years ago

If you're trying to pass through the primary GPU, try one of the single GPU passthrough guides out there. Those deal with unbinding the primary GPU. I usually just use Gigabyte motherboards and change the boot order there. Some people were able to change the GPU boot order by using the CSM compatibility mode in BIOS. I'm not sure whether that would work on an MSI motherboard though.