Issue installing GUI domain

Greetings. I’m trying to setup GPU GUI domain (sys-gui-gpu) according to official guide.

Qubes OS

Qubes release 4.2.0-alpha (R4.2)

Brief summary

I have following configuration.

Gigabyte X570 Gaming X motherboard
AMD Ryzen 7 3800X 8-Core Processor (which doesn’t have iGPU)
AMD Radeon RX 6750 XT

Steps to reproduce

sudo qubes-dom0-update --enablerepo=qubes-template-itl-testing  qubes-template-fedora-37-xfce
sudo qubesctl top.enable qvm.sys-gui-gpu
sudo qubesctl top.enable qvm.sys-gui-gpu pillar=True
sudo qubesctl --all state.highstate

Expected behavior

Normal setup

Actual behavior

[user@dom0 ~]$ sudo qubesctl --all state.highstate
/usr/lib/python3.11/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
/usr/lib/python3.11/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
[WARNING ] State for file: /etc/qubes/policy.d/45-sys-gui-gpu.policy - Neither 'source' nor 'contents' nor 'contents_pillar' nor 'contents_grains' was defined, yet 'replace' was set to 'True'. As there is no source to replace the file with, 'replace' has been set to 'False' to avoid reading the file unnecessarily.
local:
----------
          ID: sys-gui-gpu-template
    Function: qvm.template_installed
        Name: fedora-37-xfce
      Result: True
     Comment: Template fedora-37-xfce version 4.1.0 already installed
     Started: 10:31:38.121032
    Duration: 165.931 ms
     Changes:   
----------
          ID: dummy-psu-sender
    Function: pkg.installed
      Result: True
     Comment: All specified packages are already installed
     Started: 10:31:39.106974
    Duration: 278.714 ms
     Changes:   
----------
          ID: qubes-input-proxy-sender
    Function: pkg.installed
      Result: True
     Comment: All specified packages are already installed
     Started: 10:31:39.385797
    Duration: 15.374 ms
     Changes:   
----------
          ID: /etc/qubes/input-proxy-target
    Function: file.managed
      Result: True
     Comment: File /etc/qubes/input-proxy-target is in the correct state
     Started: 10:31:39.402543
    Duration: 34.36 ms
     Changes:   
----------
          ID: sys-usb-input-proxy
    Function: file.managed
        Name: /etc/qubes/policy.d/45-sys-gui-gpu.policy
      Result: True
     Comment: File /etc/qubes/policy.d/45-sys-gui-gpu.policy exists with proper permissions. No changes made.
     Started: 10:31:39.436992
    Duration: 0.844 ms
     Changes:   
----------
          ID: sys-gui-gpu
    Function: qvm.exists
      Result: True
     Comment: /usr/bin/qvm-check sys-gui-gpu None
     Started: 10:31:39.437901
    Duration: 162.575 ms
     Changes:   
----------
          ID: sys-gui-gpu-rpc
    Function: file.managed
        Name: /etc/qubes/policy.d/50-gui-sys-gui-gpu.policy
      Result: True
     Comment: File /etc/qubes/policy.d/50-gui-sys-gui-gpu.policy is in the correct state
     Started: 10:31:39.600863
    Duration: 2.196 ms
     Changes:   
----------
          ID: sys-gui-gpu-admin-local-rwx
    Function: file.append
        Name: /etc/qubes/policy.d/include/admin-local-rwx
      Result: True
     Comment: File /etc/qubes/policy.d/include/admin-local-rwx is in correct state
     Started: 10:31:39.603173
    Duration: 1.663 ms
     Changes:   
----------
          ID: sys-gui-gpu-admin-global-rwx
    Function: file.append
        Name: /etc/qubes/policy.d/include/admin-global-rwx
      Result: True
     Comment: File /etc/qubes/policy.d/include/admin-global-rwx is in correct state
     Started: 10:31:39.604911
    Duration: 1.244 ms
     Changes:   

Summary for local
------------
Succeeded: 9
Failed:    0
------------
Total states run:     9
Total run time: 662.901 ms
debian-11: SKIP (nothing to do)
fedora-37: SKIP (nothing to do)
whonix-gw-16: SKIP (nothing to do)
whonix-ws-16: SKIP (nothing to do)
/usr/lib/python3.11/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
fedora-37-xfce: OK
anon-whonix-old: SKIP (nothing to do)
backup: SKIP (nothing to do)
debian-11-dvm: SKIP (nothing to do)
default-mgmt-dvm: SKIP (nothing to do)
internet: SKIP (nothing to do)
sys-net: SKIP (nothing to do)
sys-whonix: SKIP (nothing to do)
vault-credentials: SKIP (nothing to do)
vault-gpg: SKIP (nothing to do)
vault-ssh: SKIP (nothing to do)
vault-wallets: SKIP (nothing to do)
/usr/lib/python3.11/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
vault-work: SKIP (nothing to do)
whonix-ws-16-dvm: SKIP (nothing to do)
work: SKIP (nothing to do)
sys-gui-gpu: ERROR (exit code 20, details in /var/log/qubes/mgmt-sys-gui-gpu.log)

Content of /var/log/qubes/mgmt-sys-gui-gpu.log

[user@dom0 ~]$ cat /var/log/qubes/mgmt-sys-gui-gpu.log
2023-04-26 10:32:22,846 calling 'state.highstate'...
2023-04-26 10:32:53,124 output: sys-gui-gpu:
2023-04-26 10:32:53,125 output: ----------
2023-04-26 10:32:53,125 output:           ID: /home/user/.config/autostart/nm-applet.desktop
2023-04-26 10:32:53,125 output:     Function: file.managed
2023-04-26 10:32:53,125 output:       Result: True
2023-04-26 10:32:53,125 output:      Comment: File /home/user/.config/autostart/nm-applet.desktop is in the correct state
2023-04-26 10:32:53,125 output:      Started: 10:32:52.686957
2023-04-26 10:32:53,125 output:     Duration: 48.722 ms
2023-04-26 10:32:53,125 output:      Changes:   
2023-04-26 10:32:53,125 output: ----------
2023-04-26 10:32:53,125 output:           ID: /home/user/.config/autostart/xscreensaver-autostart.desktop
2023-04-26 10:32:53,125 output:     Function: file.managed
2023-04-26 10:32:53,125 output:       Result: True
2023-04-26 10:32:53,125 output:      Comment: File /home/user/.config/autostart/xscreensaver-autostart.desktop is in the correct state
2023-04-26 10:32:53,125 output:      Started: 10:32:52.735752
2023-04-26 10:32:53,125 output:     Duration: 0.951 ms
2023-04-26 10:32:53,125 output:      Changes:   
2023-04-26 10:32:53,125 output: ----------
2023-04-26 10:32:53,125 output:           ID: /home/user/.config/autostart/qvm-start-daemon.desktop
2023-04-26 10:32:53,125 output:     Function: file.managed
2023-04-26 10:32:53,125 output:       Result: True
2023-04-26 10:32:53,125 output:      Comment: File /home/user/.config/autostart/qvm-start-daemon.desktop is in the correct state
2023-04-26 10:32:53,125 output:      Started: 10:32:52.736754
2023-04-26 10:32:53,125 output:     Duration: 0.987 ms
2023-04-26 10:32:53,125 output:      Changes:   
2023-04-26 10:32:53,126 output: ----------
2023-04-26 10:32:53,126 output:           ID: /rw/config/rc.local
2023-04-26 10:32:53,126 output:     Function: file.append
2023-04-26 10:32:53,126 output:       Result: False
2023-04-26 10:32:53,126 output:      Comment: An exception occurred in this state: Traceback (most recent call last):
2023-04-26 10:32:53,126 output:                 File "/var/tmp/.root_dd8a91_salt/pyall/salt/state.py", line 2171, in call
2023-04-26 10:32:53,126 output:                   ret = self.states[cdata["full"]](
2023-04-26 10:32:53,126 output:                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-04-26 10:32:53,126 output:                 File "/var/tmp/.root_dd8a91_salt/pyall/salt/loader.py", line 2105, in wrapper
2023-04-26 10:32:53,126 output:                   return f(*args, **kwargs)
2023-04-26 10:32:53,126 output:                          ^^^^^^^^^^^^^^^^^^
2023-04-26 10:32:53,126 output:                 File "/var/tmp/.root_dd8a91_salt/pyall/salt/states/file.py", line 6301, in append
2023-04-26 10:32:53,126 output:                   if __salt__["file.search"](
2023-04-26 10:32:53,126 output:                      ^^^^^^^^^^^^^^^^^^^^^^^^
2023-04-26 10:32:53,126 output:                 File "/var/tmp/.root_dd8a91_salt/pyall/salt/modules/file.py", line 3154, in search
2023-04-26 10:32:53,126 output:                   return replace(
2023-04-26 10:32:53,126 output:                          ^^^^^^^^
2023-04-26 10:32:53,126 output:                 File "/var/tmp/.root_dd8a91_salt/pyall/salt/modules/file.py", line 2530, in replace
2023-04-26 10:32:53,126 output:                   flags_num = _get_flags(flags)
2023-04-26 10:32:53,126 output:                               ^^^^^^^^^^^^^^^^^
2023-04-26 10:32:53,126 output:                 File "/var/tmp/.root_dd8a91_salt/pyall/salt/modules/file.py", line 1653, in _get_flags
2023-04-26 10:32:53,126 output:                   _flag = getattr(re, str(flag).upper())
2023-04-26 10:32:53,126 output:                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-04-26 10:32:53,126 output:               AttributeError: module 're' has no attribute 'RE.MULTILINE'
2023-04-26 10:32:53,126 output:      Started: 10:32:52.737792
2023-04-26 10:32:53,126 output:     Duration: 3.248 ms
2023-04-26 10:32:53,126 output:      Changes:   
2023-04-26 10:32:53,126 output: 
2023-04-26 10:32:53,126 output: Summary for sys-gui-gpu
2023-04-26 10:32:53,127 output: ------------
2023-04-26 10:32:53,127 output: Succeeded: 3
2023-04-26 10:32:53,127 output: Failed:    1
2023-04-26 10:32:53,127 output: ------------
2023-04-26 10:32:53,127 output: Total states run:     4
2023-04-26 10:32:53,127 output: Total run time:  53.908 ms
2023-04-26 10:32:53,127 exit code: 20

I searched qubes-issues by keyword re.multiline and fount this one #7891 but it’s closed saying that problem was fixed. How could I fix this?

Does your default-mgmt-dvm (or rather, its template) have https://github.com/QubesOS/updates-status/issues/3472 ? Try updating fedora-37 template first.

How could I check if default-mgmt-dvm has necessary update? My default-mgmt-dvm based on debian-11. I ran sudo dnf update in fedora-37 and fedora-37-xfce templates but it doesn't help.

I changed default-mgmt-dvm template to fedora-37 and it worked. Thanks!

Is there any extra steps for boot into sys-gui-gpu? I've added my GPU to it via qvm-pci rebooted and nothing happened.

Set sys-gui-gpu to autostart on boot.

It seems something went wrong. I've set sys-gui-gpu to autostart on boot and now my system goes to reboot after entering LUKS password. Maybe I'm attaching GPU wrong?

Here is my qvm-pci output.

[user@dom0 ~]$ qvm-pci
BACKEND:DEVID  DESCRIPTION                                                                                                                  USED BY
dom0:00_00.0   Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex                                                
dom0:00_00.2   IOMMU: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU                                                             
dom0:00_01.0   Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge                                      
dom0:00_01.1   PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge                                                   
dom0:00_01.2   PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge                                                   
dom0:00_02.0   Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge                                      
dom0:00_03.0   Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge                                      
dom0:00_03.1   PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge                                                   
dom0:00_04.0   Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge                                      
dom0:00_05.0   Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge                                      
dom0:00_07.0   Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge                                      
dom0:00_07.1   PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]                       
dom0:00_08.0   Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge                                      
dom0:00_08.1   PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]                       
dom0:00_14.0   SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller                                                               
dom0:00_14.3   ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge                                                                
dom0:00_18.0   Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 0                          
dom0:00_18.1   Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 1                          
dom0:00_18.2   Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 2                          
dom0:00_18.3   Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 3                          
dom0:00_18.4   Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 4                          
dom0:00_18.5   Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 5                          
dom0:00_18.6   Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 6                          
dom0:00_18.7   Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 7                          
dom0:01_00.0   Non-Volatile memory controller: Phison Electronics Corporation E12 NVMe Controller                                           
dom0:02_00.0   PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream                                                       
dom0:03_02.0   PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge                                                       
dom0:03_04.0   PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge                                                       
dom0:03_08.0   PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge                                                       
dom0:03_09.0   PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge                                                       
dom0:03_0a.0   PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge                                                       
dom0:04_00.0   Non-Volatile memory controller: Lite-On Technology Corporation                                                               
dom0:05_00.0   Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller               sys-net
dom0:06_00.0   Non-Essential Instrumentation: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP                              
dom0:06_00.1   USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller                                           sys-usb (no-strict-reset=True)
dom0:06_00.3   USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller                                           sys-usb (no-strict-reset=True)
dom0:07_00.0   SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode]                                          
dom0:08_00.0   SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode]                                          
dom0:09_00.0   PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch                            
dom0:0a_00.0   PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch                          
dom0:0b_00.0   VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 22 [Radeon RX 6700/6700 XT/6750 XT / 6800M/6850M XT]  
dom0:0b_00.1   Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21/23 HDMI/DP Audio Controller                                     
dom0:0c_00.0   Non-Essential Instrumentation: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function                       
dom0:0d_00.0   Non-Essential Instrumentation: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP                              
dom0:0d_00.1   Encryption controller: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP                  
dom0:0d_00.3   USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller                                           sys-usb (no-strict-reset=True)
dom0:0d_00.4   Audio device: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller

I'm attaching it like this.

qvm-pci a sys-gui-gpu dom0:0b_00.0 --persistent -o permissive=true

I think you did that correctly, the crash may mean your GPU is not compatible with PCI passthrough (or rather: this specific case is not supported by the driver). Generally, it's hard to say more without log message from that crash (maybe it survived in /var/lib/systemd/pstore?), but I have one more idea to try: add nomodeset to dom0 kernel cmdline.

I checked /var/lib/systemd/pstore/ in dom0 and it's empty.

[user@dom0 ~]$ ls -lah /var/lib/systemd/pstore/
total 8.0K
drwxr-xr-x 2 root root 4.0K Apr 24 15:46 .
drwxr-xr-x 7 root root 4.0K Apr 24 12:58 ..

I tried to add nomodeset option on boot right after rhgb quiet. System booted but there was just blinking _ in the upper left corner. I switched to tty2 by pressing CTRL + ALT + F2 and checked VMs by qvm-ls but sys-gui-gpu was Halted. I tried to run it with qvm-run sys-gui-gpu but nothing happened.

@aon4oor I've got notification about your comment, but I don't see it here. Anyway, use qvm-start sys-gui-gpu for starting.

I lost access to my account and created a new one but github banned it without a reason :/ Nevermind.

I tried qvm-start sys-gui-gpu and it thrown following error:

Start failed: internal error: Unable to reset PCI device 0000:0b:00.0: internal error: Active 0000:0b:00.1 devices on bus with 0000:0b:00.0, not doing bus reset, see /var/log/libvirt/libxl/libxl-driver.log

Contents of /var/log/libvirt/libxl/libxl-driver.log

[user@dom0 ~]$ sudo cat /var/log/libvirt/libxl/libxl-driver.log
2023-04-24 10:19:25.104+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-25 04:33:33.386+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-25 04:34:43.342+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-25 05:08:55.890+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-25 05:10:03.511+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-25 09:54:02.285+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-25 09:55:20.908+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-26 06:26:01.311+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-26 06:27:08.285+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-26 06:32:46.696+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-26 06:37:26.658+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-26 06:56:44.113+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-26 09:57:35.584+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-26 10:11:55.289+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-27 12:48:04.824+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-27 12:49:14.070+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-27 13:00:22.887+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-27 13:31:31.608+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-27 13:40:10.329+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-28 04:49:11.072+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-28 04:51:16.247+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-28 04:57:16.212+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-28 05:19:14.796+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-28 06:52:44.941+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-28 06:53:54.454+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-29 05:53:23.634+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-29 05:55:19.067+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-29 05:57:02.946+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1
2023-04-29 05:58:07.712+0000: libxl: libxl_pci.c:1585:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:06:00.1

You may need to assign 0000:0b:00.1 (audio function of that graphics card, for audio via HDMI) to the GUI qube too. Alternatively, add also -o no-strict-reset=True to that qvm-pci command to ignore this issue, but there is no guarantee it won't explode later.

I assigned audio function like this

qvm-pci a sys-gui-gpu dom0:0b_00.1 --persistent -o permissive=true

I set sys-gui-gpu to autostart on boot and added nomodeset to dom0 kernel cmdline. Now system boots up with black screen and I cannot switch to different tty. Without nomodeset system goes to reboot.

I think I'm booting into sys-gui-gpu but something goes wrong. Which log files I can check and provide here to debug this?

@marmarek any chance get this debugged?

I am also experiencing problems, after set up of 'sys-gui-gpu'. This seems to happen when attaching the pci device(s) to the qube. The system does a hard reset after the qube starts or after decrypting.

Hardware: I am running on a laptop, with Intel HD Graphics and discrete Nvidia GPU. ~~Using no-strict-reset~~=~~True has no effect~~

You can revert changes:

If you set sys-gui to autostart on boot, you can bypass this and get back into the system, by adding qubes.skip_autostart to the GRUB kernel cmdline (press 'e' to edit boot menu and add after 'quiet').
If you are using nomodeset or the screen remains black, you are likely logged in, but with no video output. Launch a terminal by typing Alt+F2 then xfce4-terminal and press enter. Then you can execute qvm-pci detach sys-gui-gpu dom0:[BDF] and type reboot. If you don't remember the BDF id, then you could run this. /Display/ will perform a regex search for the the device type of the pci device to detach, 'Display Controller' but you may also want to do /VGA/. -- Type it slowly. If you think you made a mistake hit Ctrl+C and start again. Otherwise you may execute the wrong command.

BDF=`lspci | awk '/Display/ {print $1}'` && qvm-pci detach sys-gui-gpu dom0:$BDF

EDIT: I have managed to now successfully boot into sys-gui-gpu on system boot, running my laptops discrete Nvidia 1070 graphics card (GP104BM), displayed on an external monitor. -- the caveat is that I have to enter LUKS password and use tty console blindly.

Here is what I did.

After attaching the pci device(s), I had to hide them from dom0 in /boot/efi/EFI/grub.cfg adding this option flag in the kernel cmdline: rd.qubes.hide_pci=01:00.0,01:00.1 and regenerate grub menu grub2-mkconfig -o /boot/efi/EFI/grub.cfg.
Then the crucial part, thanks to @marmarek for mentioning in another post, I had to prevent X11 from starting when sys-gui-gpu autostarts. This was accomplished by doing in dom0 systemctl disable lightdm.service and qvm-prefs sys-gui-gpu autostart True.
I then did qubes-prefs default_guivm sys-gui-gpu as there was likely a bug present where it had not done this.
I also had to manually set guivm property for all VMs to sys-gui-gpu, again likely because a bug in the salt code

Side note: If you follow this procedure and it works for you, to get back into dom0 you need to first shutdown 'sys-gui-gpu', and type commands into a tty console blindly (at least for me anyway). Do as follows: CTRL+ALT+F2 to make sure you're in a tty console, enter your dom0 username, enter your dom0 password then type startxfce4 presuming dom0 is using xfce.

QubesOS / qubes-issues