Open aon4oor opened 1 year ago
Does your default-mgmt-dvm (or rather, its template) have https://github.com/QubesOS/updates-status/issues/3472 ? Try updating fedora-37 template first.
How could I check if default-mgmt-dvm
has necessary update?
My default-mgmt-dvm
based on debian-11.
I ran sudo dnf update
in fedora-37
and fedora-37-xfce
templates but it doesn't help.
I changed default-mgmt-dvm
template to fedora-37
and it worked.
Thanks!
Is there any extra steps for boot into sys-gui-gpu
?
I've added my GPU to it via qvm-pci
rebooted and nothing happened.
Set sys-gui-gpu to autostart on boot.
It seems something went wrong.
I've set sys-gui-gpu
to autostart on boot and now my system goes to reboot after entering LUKS password.
Maybe I'm attaching GPU wrong?
Here is my qvm-pci
output.
[user@dom0 ~]$ qvm-pci
BACKEND:DEVID DESCRIPTION USED BY
dom0:00_00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex
dom0:00_00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU
dom0:00_01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
dom0:00_01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge
dom0:00_01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge
dom0:00_02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
dom0:00_03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
dom0:00_03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge
dom0:00_04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
dom0:00_05.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
dom0:00_07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
dom0:00_07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]
dom0:00_08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
dom0:00_08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]
dom0:00_14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller
dom0:00_14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge
dom0:00_18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 0
dom0:00_18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 1
dom0:00_18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 2
dom0:00_18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 3
dom0:00_18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 4
dom0:00_18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 5
dom0:00_18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 6
dom0:00_18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 7
dom0:01_00.0 Non-Volatile memory controller: Phison Electronics Corporation E12 NVMe Controller
dom0:02_00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream
dom0:03_02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
dom0:03_04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
dom0:03_08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
dom0:03_09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
dom0:03_0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
dom0:04_00.0 Non-Volatile memory controller: Lite-On Technology Corporation
dom0:05_00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller sys-net
dom0:06_00.0 Non-Essential Instrumentation: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
dom0:06_00.1 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller sys-usb (no-strict-reset=True)
dom0:06_00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller sys-usb (no-strict-reset=True)
dom0:07_00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode]
dom0:08_00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode]
dom0:09_00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch
dom0:0a_00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch
dom0:0b_00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]
dom0:0b_00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21/23 HDMI/DP Audio Controller
dom0:0c_00.0 Non-Essential Instrumentation: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function
dom0:0d_00.0 Non-Essential Instrumentation: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
dom0:0d_00.1 Encryption controller: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP
dom0:0d_00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller sys-usb (no-strict-reset=True)
dom0:0d_00.4 Audio device: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller
I'm attaching it like this.
qvm-pci a sys-gui-gpu dom0:0b_00.0 --persistent -o permissive=true
I think you did that correctly, the crash may mean your GPU is not compatible with PCI passthrough (or rather: this specific case is not supported by the driver). Generally, it's hard to say more without log message from that crash (maybe it survived in /var/lib/systemd/pstore
?), but I have one more idea to try: add nomodeset
to dom0 kernel cmdline.
I checked /var/lib/systemd/pstore/
in dom0
and it's empty.
[user@dom0 ~]$ ls -lah /var/lib/systemd/pstore/
total 8.0K
drwxr-xr-x 2 root root 4.0K Apr 24 15:46 .
drwxr-xr-x 7 root root 4.0K Apr 24 12:58 ..
I tried to add nomodeset
option on boot right after rhgb quiet
. System booted but there was just blinking _
in the upper left corner. I switched to tty2
by pressing CTRL + ALT + F2
and checked VMs by qvm-ls
but sys-gui-gpu
was Halted
.
I tried to run it with qvm-run sys-gui-gpu
but nothing happened.
@aon4oor I've got notification about your comment, but I don't see it here. Anyway, use qvm-start sys-gui-gpu
for starting.
I lost access to my account and created a new one but github banned it without a reason :/ Nevermind.
I tried qvm-start sys-gui-gpu
and it thrown following error:
Start failed: internal error: Unable to reset PCI device 0000:0b:00.0: internal error: Active 0000:0b:00.1 devices on bus with 0000:0b:00.0, not doing bus reset, see /var/log/libvirt/libxl/libxl-driver.log
Contents of /var/log/libvirt/libxl/libxl-driver.log
[user@dom0 ~]$ sudo cat /var/log/libvirt/libxl/libxl-driver.log
2023-04-24 10:19:25.104+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-25 04:33:33.386+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-25 04:34:43.342+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-25 05:08:55.890+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-25 05:10:03.511+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-25 09:54:02.285+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-25 09:55:20.908+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-26 06:26:01.311+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-26 06:27:08.285+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-26 06:32:46.696+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-26 06:37:26.658+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-26 06:56:44.113+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-26 09:57:35.584+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-26 10:11:55.289+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-27 12:48:04.824+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-27 12:49:14.070+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-27 13:00:22.887+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-27 13:31:31.608+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-27 13:40:10.329+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-28 04:49:11.072+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-28 04:51:16.247+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-28 04:57:16.212+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-28 05:19:14.796+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-28 06:52:44.941+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-28 06:53:54.454+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-29 05:53:23.634+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-29 05:55:19.067+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-29 05:57:02.946+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-29 05:58:07.712+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
You may need to assign 0000:0b:00.1 (audio function of that graphics card, for audio via HDMI) to the GUI qube too.
Alternatively, add also -o no-strict-reset=True
to that qvm-pci command to ignore this issue, but there is no guarantee it won't explode later.
I assigned audio function like this
qvm-pci a sys-gui-gpu dom0:0b_00.1 --persistent -o permissive=true
I set sys-gui-gpu
to autostart on boot and added nomodeset
to dom0 kernel cmdline.
Now system boots up with black screen and I cannot switch to different tty
.
Without nomodeset
system goes to reboot.
I think I'm booting into sys-gui-gpu
but something goes wrong.
Which log files I can check and provide here to debug this?
@marmarek any chance get this debugged?
I am also experiencing problems, after set up of 'sys-gui-gpu'. This seems to happen when attaching the pci device(s) to the qube. The system does a hard reset after the qube starts or after decrypting.
Hardware:
I am running on a laptop, with Intel HD Graphics and discrete Nvidia GPU. Using no-strict-reset=True has no effect
You can revert changes:
qubes.skip_autostart
to the GRUB kernel cmdline (press 'e' to edit boot menu and add after 'quiet').Alt+F2
then xfce4-terminal
and press enter
. Then you can execute qvm-pci detach sys-gui-gpu dom0:[BDF]
and type reboot
. If you don't remember the BDF id, then you could run this. /Display/ will perform a regex search for the the device type of the pci device to detach, 'Display Controller' but you may also want to do /VGA/. -- Type it slowly. If you think you made a mistake hit Ctrl+C and start again. Otherwise you may execute the wrong command.BDF=`lspci | awk '/Display/ {print $1}'` && qvm-pci detach sys-gui-gpu dom0:$BDF
EDIT: I have managed to now successfully boot into sys-gui-gpu on system boot, running my laptops discrete Nvidia 1070 graphics card (GP104BM), displayed on an external monitor. -- the caveat is that I have to enter LUKS password and use tty console blindly.
Here is what I did.
/boot/efi/EFI/grub.cfg
adding this option flag in the kernel cmdline: rd.qubes.hide_pci=01:00.0,01:00.1
and regenerate grub menu grub2-mkconfig -o /boot/efi/EFI/grub.cfg
.systemctl disable lightdm.service
and qvm-prefs sys-gui-gpu autostart True
.qubes-prefs default_guivm sys-gui-gpu
as there was likely a bug present where it had not done this.Side note:
If you follow this procedure and it works for you, to get back into dom0 you need to first shutdown 'sys-gui-gpu', and type commands into a tty console blindly (at least for me anyway). Do as follows: CTRL+ALT+F2
to make sure you're in a tty console, enter your dom0 username
, enter your dom0 password
then type startxfce4
presuming dom0 is using xfce.
Greetings. I’m trying to setup GPU GUI domain (sys-gui-gpu) according to official guide.
Qubes OS
Qubes release 4.2.0-alpha (R4.2)
Brief summary
I have following configuration.
Steps to reproduce
Expected behavior
Normal setup
Actual behavior
Content of
/var/log/qubes/mgmt-sys-gui-gpu.log
I searched qubes-issues by keyword
re.multiline
and fount this one #7891 but it’s closed saying that problem was fixed. How could I fix this?