Closed bytemaestr0 closed 5 months ago
hello @bytemaestr0 can you provide the output from utils/ls-iommu -grk
and post that here along with the debug.log
.
the utils
folder is inside the folder that has the quickpassthrough
binary
If you do not have the ls-iommu
binary just start quickpassthrough
once
note: also i updated your message for readability by using code blocks ``` put text here to put in the code block ``` will result in
put text here to put in the code block
the output of utils/ls-iommu -grk is:
IOMMU Group 12: 0000:01:00.0 USB controller [0c03] Advanced Micro Devices, Inc. [AMD] unknown [1022:43d0] (rev 01)
Subsystem: ASMedia Technology Inc. unknown [1b21:1142]
Kernel driver in use: xhci_hcd
IOMMU Group 12: 0000:01:00.1 SATA controller [0106] Advanced Micro Devices, Inc. [AMD] 400 Series Chipset SATA Controller [1022:43c8] (rev 01)
Subsystem: ASMedia Technology Inc. 400 Series Chipset SATA Controller [1b21:1062]
Kernel driver in use: ahci
IOMMU Group 12: 0000:01:00.2 PCI bridge [0604] Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Bridge [1022:43c6] (rev 01)
Subsystem: ASMedia Technology Inc. 400 Series Chipset PCIe Bridge [1b21:0201]
Kernel driver in use: pcieport
IOMMU Group 12: 0000:02:00.0 PCI bridge [0604] Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
Subsystem: ASMedia Technology Inc. 400 Series Chipset PCIe Port [1b21:3306]
Kernel driver in use: pcieport
IOMMU Group 12: 0000:02:01.0 PCI bridge [0604] Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
Subsystem: ASMedia Technology Inc. 400 Series Chipset PCIe Port [1b21:3306]
Kernel driver in use: pcieport
IOMMU Group 12: 0000:02:02.0 PCI bridge [0604] Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
Subsystem: ASMedia Technology Inc. 400 Series Chipset PCIe Port [1b21:3306]
Kernel driver in use: pcieport
IOMMU Group 12: 0000:02:03.0 PCI bridge [0604] Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
Subsystem: ASMedia Technology Inc. 400 Series Chipset PCIe Port [1b21:3306]
Kernel driver in use: pcieport
IOMMU Group 12: 0000:02:04.0 PCI bridge [0604] Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
Subsystem: ASMedia Technology Inc. 400 Series Chipset PCIe Port [1b21:3306]
Kernel driver in use: pcieport
IOMMU Group 12: 0000:02:09.0 PCI bridge [0604] Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
Subsystem: ASMedia Technology Inc. 400 Series Chipset PCIe Port [1b21:3306]
Kernel driver in use: pcieport
IOMMU Group 12: 0000:03:00.0 Ethernet controller [0200] Realtek Semiconductor Co., Ltd. RTL8111/8168/8211/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15)
Subsystem: ASUSTeK Computer Inc. PRIME B450M-A Motherboard [1043:8677]
Kernel driver in use: r8168
IOMMU Group 12: 0000:07:00.0 VGA compatible controller [0300] NVIDIA Corporation GA106 [GeForce RTX 3060 Lite Hash Rate] [10de:2504] (rev a1)
Subsystem: Micro-Star International Co., Ltd. [MSI] GA106 [GeForce RTX 3060 Lite Hash Rate] [1462:397d]
Kernel driver in use: vfio-pci
IOMMU Group 12: 0000:07:00.1 Audio device [0403] NVIDIA Corporation GA106 High Definition Audio Controller [10de:228e] (rev a1)
Subsystem: Micro-Star International Co., Ltd. [MSI] GA106 High Definition Audio Controller [1462:397d]
Kernel driver in use: snd_hda_intel
IOMMU Group 12: 0000:08:00.0 Non-Volatile memory controller [0108] Kingston Technology Company, Inc. A2000 NVMe SSD SM2263EN [2646:2263] (rev 03)
Subsystem: Kingston Technology Company, Inc. A2000 NVMe SSD SM2263EN [2646:2263]
Kernel driver in use: nvme
IOMMU Group 13: 0000:09:00.0 VGA compatible controller [0300] Advanced Micro Devices, Inc. [AMD/ATI] Baffin [Radeon RX 550 640SP / RX 560/560X] [1002:67ff] (rev cf)
Subsystem: Sapphire Technology Limited Radeon RX 560 [1da2:e348]
Kernel driver in use: amdgpu
IOMMU Group 13: 0000:09:00.1 Audio device [0403] Advanced Micro Devices, Inc. [AMD/ATI] Baffin HDMI/DP Audio [Radeon RX 550 640SP / RX 560/560X] [1002:aae0]
Subsystem: Sapphire Technology Limited Baffin HDMI/DP Audio [Radeon RX 550 640SP / RX 560/560X] [1da2:aae0]
Kernel driver in use: snd_hda_intel
the debug.log is:
2024/03/24 16:59:31
/etc/modprobe.d found on the system
Creating config/etc/modprobe.d
2024/03/24 16:59:31
/etc/default found on the system
Creating config/etc/default
2024/03/24 16:59:31
/etc/dracut.conf.d found on the system
Creating config/etc/dracut.conf.d
2024/03/24 16:59:31
/etc/default/grub found on the system
Creating config/etc/default/grub
2024/03/24 16:59:31
Executing: utils/ls-iommu -g -F vendor:,prod_name,optional_revision:,device_id
2024/03/24 16:59:31
ls-iommu query returned
IOMMU Group 12: NVIDIA Corporation: GA106 [GeForce RTX 3060 Lite Hash Rate] (rev a1): [10de:2504]
IOMMU Group 13: Advanced Micro Devices, Inc. [AMD/ATI]: Baffin [Radeon RX 550 640SP / RX 560/560X] (rev cf): [1002:67ff]
2024/03/24 16:59:34
Executing: utils/ls-iommu -g -r -i 12 -F vendor:,prod_name,optional_revision:,device_id
2024/03/24 16:59:36
ls-iommu query returned
IOMMU Group 12: Advanced Micro Devices, Inc. [AMD]: 400 Series Chipset PCIe Bridge (rev 01): [1022:43c6]
IOMMU Group 12: Advanced Micro Devices, Inc. [AMD]: 400 Series Chipset PCIe Port (rev 01): [1022:43c7]
IOMMU Group 12: Advanced Micro Devices, Inc. [AMD]: 400 Series Chipset SATA Controller (rev 01): [1022:43c8]
IOMMU Group 12: Advanced Micro Devices, Inc. [AMD]: unknown (rev 01): [1022:43d0]
IOMMU Group 12: Kingston Technology Company, Inc.: A2000 NVMe SSD SM2263EN (rev 03): [2646:2263]
IOMMU Group 12: NVIDIA Corporation: GA106 High Definition Audio Controller (rev a1): [10de:228e]
IOMMU Group 12: NVIDIA Corporation: GA106 [GeForce RTX 3060 Lite Hash Rate] (rev a1): [10de:2504]
IOMMU Group 12: Realtek Semiconductor Co., Ltd.: RTL8111/8168/8211/8411 PCI Express Gigabit Ethernet Controller (rev 15): [10ec:8168]
2024/03/24 16:59:45
Executing: utils/ls-iommu -g -rr -i 12 -F vendor:,prod_name,optional_revision:,device_id
2024/03/24 16:59:48
ls-iommu query returned
IOMMU Group 12: Advanced Micro Devices, Inc. [AMD]: 400 Series Chipset PCIe Bridge (rev 01): [1022:43c6]
IOMMU Group 12: Advanced Micro Devices, Inc. [AMD]: 400 Series Chipset PCIe Port (rev 01): [1022:43c7]
IOMMU Group 12: Advanced Micro Devices, Inc. [AMD]: 400 Series Chipset SATA Controller (rev 01): [1022:43c8]
IOMMU Group 12: Advanced Micro Devices, Inc. [AMD]: unknown (rev 01): [1022:43d0]
IOMMU Group 12: Kingston Technology Company, Inc.: A2000 NVMe SSD SM2263EN (rev 03): [2646:2263]
IOMMU Group 12: NVIDIA Corporation: GA106 High Definition Audio Controller (rev a1): [10de:228e]
IOMMU Group 12: NVIDIA Corporation: GA106 [GeForce RTX 3060 Lite Hash Rate] (rev a1): [10de:2504]
IOMMU Group 12: Realtek Semiconductor Co., Ltd.: RTL8111/8168/8211/8411 PCI Express Gigabit Ethernet Controller (rev 15): [10ec:8168]
2024/03/24 17:00:12
Executing: utils/ls-iommu -g -i 12 --rom
2024/03/24 17:00:12
ls-iommu query returned
/sys/devices/pci0000:00/0000:00:01.3/0000:01:00.2/0000:02:04.0/0000:07:00.0/rom
2024/03/24 17:00:12
Writing utils/dump_vbios.sh
2024/03/24 17:00:21
Adding vfio_pci.disable_vga=1 to config/kernel_args
2024/03/24 17:00:21
User has disabled vfio video output on host, adding disable_vga=1 to the optional hardcoded vfio_pci options
2024/03/24 17:00:21
Writing config/etc/modprobe.d/vfio.conf:
## This is an autogenerated file that stubs your graphic card for use with vfio
## This file should be placed inside /etc/modprobe.d/
# Uncomment the line below to "hardcode" your graphic card to be bound to the vfio-pci driver.
# In most cases this should not be neccessary, it will also prevent you from turning off vfio in the bootloader.
#options vfio_pci ids=10de:228e,10de:2504 disable_vga=1
# Make sure vfio_pci is loaded before these modules: nvidia, nouveau, amdgpu and radeon
softdep nvidia pre: vfio vfio_pci
softdep nouveau pre: vfio vfio_pci
softdep amdgpu pre: vfio vfio_pci
softdep radeon pre: vfio vfio_pci
2024/03/24 17:00:21
Writing to config/etc/dracut.conf.d/vfio.conf:
add_drivers+=" vfio_pci vfio vfio_iommu_type1 "
2024/03/24 17:00:21
Configuring grub2 manually
2024/03/24 17:00:21
Read /etc/default/grub:
# GRUB boot loader configuration
GRUB_DEFAULT='0'
GRUB_TIMEOUT='5'
GRUB_DISTRIBUTOR='EndeavourOS'
GRUB_CMDLINE_LINUX_DEFAULT='nowatchdog nvme_load=YES resume=UUID=2b98b356-0833-443f-8ce8-d6800e9cb6ac loglevel=3'
GRUB_CMDLINE_LINUX="iommu=pt amd_iommu=on vfio_pci.ids=10de:2504,10de:228e vfio_pci.disable_vga=1 rd.driver.pre=vfio_pci"
# Preload both GPT and MBR modules so that they are not missed
GRUB_PRELOAD_MODULES="part_gpt part_msdos"
# Uncomment to enable booting from LUKS encrypted devices
#GRUB_ENABLE_CRYPTODISK=y
# Set to 'countdown' or 'hidden' to change timeout behavior,
# press ESC key to display menu.
GRUB_TIMEOUT_STYLE=menu
# Uncomment to use basic console
GRUB_TERMINAL_INPUT=console
# Uncomment to disable graphical terminal
#GRUB_TERMINAL_OUTPUT=console
# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `videoinfo'
GRUB_GFXMODE=auto
# Uncomment to allow the kernel use the same resolution used by grub
GRUB_GFXPAYLOAD_LINUX=keep
# Uncomment if you want GRUB to pass to the Linux kernel the old parameter
# format "root=/dev/xxx" instead of "root=/dev/disk/by-uuid/xxx"
#GRUB_DISABLE_LINUX_UUID=true
# Uncomment to disable generation of recovery mode menu entries
GRUB_DISABLE_RECOVERY='true'
# Uncomment and set to the desired menu colors. Used by normal and wallpaper
# modes only. Entries specified as foreground/background.
#GRUB_COLOR_NORMAL="light-blue/black"
#GRUB_COLOR_HIGHLIGHT="light-cyan/blue"
# Uncomment one of them for the gfx desired, a image background or a gfxtheme
GRUB_BACKGROUND='/usr/share/endeavouros/splash.png'
#GRUB_THEME="/path/to/gfxtheme"
# Uncomment to get a beep at GRUB start
#GRUB_INIT_TUNE="480 440 1"
# Uncomment to make GRUB remember the last selection. This requires
# setting 'GRUB_DEFAULT=saved' above.
#GRUB_SAVEDEFAULT=true
# Uncomment to disable submenus in boot menu
GRUB_DISABLE_SUBMENU='false'
# Probing for other operating systems is disabled for security reasons. Read
# documentation on GRUB_DISABLE_OS_PROBER, if still want to enable this
# functionality install os-prober and uncomment to detect and include other
# operating systems.
GRUB_DISABLE_OS_PROBER=false
2024/03/24 17:00:21
Replacing line in config/etc/default/grub:
GRUB_CMDLINE_LINUX="iommu=pt amd_iommu=on vfio_pci.ids=10de:2504,10de:228e vfio_pci.disable_vga=1 rd.driver.pre=vfio_pci"
With:
GRUB_CMDLINE_LINUX="iommu=pt amd_iommu=on vfio_pci.ids=10de:228e,10de:2504 vfio_pci.disable_vga=1 rd.driver.pre=vfio_pci"
2024/03/24 17:00:27
Applying grub2 changes
2024/03/24 17:00:27
Executing command:
sudo cp -v "config/etc/default/grub" /etc/default/grub
2024/03/24 17:00:27
'config/etc/default/grub' -> '/etc/default/grub'
2024/03/24 17:00:29
Generating grub configuration file ...
Found background: /usr/share/endeavouros/splash.png
Found linux image: /boot/vmlinuz-linux-lts
Found initrd image: /boot/amd-ucode.img /boot/initramfs-linux-lts.img
Found fallback initrd image(s) in /boot: amd-ucode.img initramfs-linux-lts-fallback.img
Found linux image: /boot/vmlinuz-linux
Found initrd image: /boot/amd-ucode.img /boot/initramfs-linux.img
Found fallback initrd image(s) in /boot: amd-ucode.img initramfs-linux-fallback.img
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
Adding boot menu entry for UEFI Firmware Settings ...
done
2024/03/24 17:00:29
'config/etc/modprobe.d/vfio.conf' -> '/etc/modprobe.d/vfio.conf'
2024/03/24 17:00:29
'config/etc/dracut.conf.d/vfio.conf' -> '/etc/dracut.conf.d/vfio'
2024/03/24 17:00:29
Executing: sudo dracut -f -v --kver 6.6.22-1-lts
2024/03/24 17:00:29
dracut: Executing: /usr/bin/dracut -f -v --kver 6.6.22-1-lts
dracut: Can't write to /boot/efi/843f973e33514afdb46423d177c1bd29/6.6.22-1-lts: Directory /boot/efi/843f973e33514afdb46423d177c1bd29/6.6.22-1-lts does not exist or is not accessible.
it seems that the problem might have something to do with dracut but im not an expert.
Well this is easy, the card you have selected will never work in the PCIe slot it is in, because it is in IOMMU group 12 which is shared with a bunch of other stuff, not sure why the audio controller doesnt bind to vfio-pci, but it might be related?
either way you might have better luck if you swap the nvidia card to use the pcie slot that the amd card is using.
The dialog where you select the gpu clearly states that the list should only contain the GPU and anything associated with it.
i mean i thought that if i went over to advanced, then manual select, it would somehow isolate those 2 from iommu group. that was my mistake. im going to look for a way to set it to another group, thanks for all the help!
i mean i thought that if i went over to advanced, then manual select, it would somehow isolate those 2 from iommu group. that was my mistake. im going to look for a way to set it to another group, thanks for all the help!
no the advanced one is for situations where the other stuff in the group is fine to be passed through, but you shouldnt bind it to vfio-pci
or in situations where you have an intel ARC card which will share the vendor ID with intel chipsets and the intel cpu and anything else from intel (which causes the relative checks to be unreliable, so you have to supply it manually then) 😄
i have 2 gpus and virtualization enabled, so it should work this is the output of
also, i checked dmesg and it showed that both devices are being added:
(they are both correct, i triple checked) and also in
/etc/modprobe.d/vfio.conf
i also added optionsvfio-pci ids=10de:2504,10de:228e
but it didnt work either