Gamescope segfault on embeded mode

[cylian914]

Hi, im curently trying to run gamescope on the tty, in embeded mode but i can't get it to work. Im using a hybrid laptop with a GTX950M, im using nvidia 545.29.06 driver (KMS). Here the log

Gamescope -w 1920 -h 900 -W 1600 -H 1080 -e --prefer-vk-device 01:00.0  -- steam

wlserver: [backend/headless/backend.c:68] Creating headless backend
wlserver: [libseat] [libseat/libseat.c:73] Seat opened with backend 'seatd'
wlserver: [libseat] [libseat/backend/seatd.c:212] Enabling seat
wlserver: [backend/session/session.c:109] Successfully loaded libseat session
drm: warning: picking an arbitrary DRM device
drm: Immediate flips are not supported by the KMS driver
drm: found new connector 'DP-1'
drm: found new connector 'HDMI-A-1'
drm: Connector make AUO model (null)
drm: [colorimetry]: EDID with colorimetry detected. Using it
drm: [colorimetry]: r 0.620117 0.349609
drm: [colorimetry]: g 0.315430 0.615234
drm: [colorimetry]: b 0.150391 0.113281
drm: [colorimetry]: w 0.313477 0.329102
drm: found new connector 'eDP-1'
drm: CRTC 51 has no VALVE1_CRTC_REGAMMA_TF support
drm: CRTC 72 has no VALVE1_CRTC_REGAMMA_TF support
drm: CRTC 93 has no VALVE1_CRTC_REGAMMA_TF support
drm: Connectors:
drm:   DP-1 (disconnected)
drm:   HDMI-A-1 (disconnected)
drm:   eDP-1 (connected)
drm: selecting connector eDP-1
drm: selecting mode 1600x900@60Hz
vulkan: selecting physical device 'NVIDIA GeForce GTX 950M': queue family 0
vulkan: physical device supports DRM format modifiers
vulkan: The vulkan driver does not support foreign queues, disabling modifier support.
vkCreateDevice failed with a high-priority queue. Falling back to regular priority.
vulkan: supported DRM formats for sampling usage:
vulkan:   AR24 (0x34325241)
vulkan:   XR24 (0x34325258)
vulkan:   AB24 (0x34324241)
vulkan:   XB24 (0x34324258)
vulkan:   NV12 (0x3231564E)
vulkan:   XB4H (0x48344258)
vulkan:   XB30 (0x30334258)
vulkan:   XR30 (0x30335258)
vulkan: drm_fbid_from_dmabuf failed
vulkan: failed to allocate buffer for KMS
vulkan_make_output failed
[1]    14109 segmentation fault (core dumped)  gamescope -w 1920 -h 900 -W 1600 -H 1080 -e --prefer-vk-device 01:00.0 -- 

i can make it work on dwl then running gamescope in nested but i still get an error when running steam:

SteamUpdateUI: An X Error occurred
X Error of failed request:  BadAlloc (insufficient resources for operation)
Major opcode of failed request:  150 (GLX)
Minor opcode of failed request:  5 (X_GLXMakeCurrent)
Serial number of failed request:  0
xerror_handler: X failed, continuing
src/steamexe/updateui_xwin.cpp (1543) : UpdateUI GL not available

running https://github.com/sharkautarch/gamescope/tree/nvidia-fix also doesn't work:

vulkan: The vulkan driver does not support foreign queues, disabling modifier support.
vkCreateDevice failed with a high-priority queue (general + compute). Falling back to regular priority (general).
vkCreateDevice failed with a high-priority queue (compute). Falling back to regular priority (all).
[1]    14253 segmentation fault (core dumped)  ./gamescope -w 1920 -h 900 -W 1600 -H 1080 -e --prefer-vk-device 01:00.0 -- 

Does anyone know what the issue is or how to fix it ?

[sharkautarch]

running https://github.com/sharkautarch/gamescope/tree/nvidia-fix also doesn't work:

vulkan: The vulkan driver does not support foreign queues, disabling modifier support. vkCreateDevice failed with a high-priority queue (general + compute). Falling back to regular priority (general). vkCreateDevice failed with a high-priority queue (compute). Falling back to regular priority (all).

I just doubled check, and there is no code in my nvidia-fix branch that would print the part of the above output with (general + compute) or (general) or (compute) or (all). ... oh wait... I think you might have accidentily switched up the outputs of ValveSoftware/gamescope/master and sharkautarch/gamescope/nvidia-fix ...

I've heard some other people have issues with running my nvidia-fix branch of gamescope in embedded mode... try running: (again specific to nvidia-fix) ENABLE_GAMESCOPE_WSI=0 gamescope -w 1920 -h 900 -W 1600 -H 1080 -e --prefer-vk-device 01:00.0 -- steam

Also, if you experience a lot of stuttering w/ running the above command (or with any use of gamescope w/ nvidia), you might want to use my nvidia-fix-and-vblank-debug-extra-experimental-v4.8 branch

[cylian914]

Sorry for the late response, i didn't have the time to test it.

After switching up to the right branch it still crashes.

Looking at the dumb file it look like its the driver who crash:

                Stack trace of thread 57594:
                #0  0x00007ff5bfcb3184 n/a (libnvidia-glvkspirv.so.545.29.06 + 0x2b3184)
                #1  0x00007ff5bfcb4267 n/a (libnvidia-glvkspirv.so.545.29.06 + 0x2b4267)
                #2  0x00007ff5bfb3bf21 n/a (libnvidia-glvkspirv.so.545.29.06 + 0x13bf21)
                #3  0x00007ff5bfb26c96 _nv013nvvm (libnvidia-glvkspirv.so.545.29.06 + 0x126c96)
                #4  0x00007ff5c1541b31 n/a (libnvidia-eglcore.so.545.29.06 + 0xd41b31)
                #5  0x00007ff5c1542cc0 n/a (libnvidia-eglcore.so.545.29.06 + 0xd42cc0)
                #6  0x00007ff5c1545c04 n/a (libnvidia-eglcore.so.545.29.06 + 0xd45c04)
                #7  0x00007ff5c1545f4b n/a (libnvidia-eglcore.so.545.29.06 + 0xd45f4b)
                #8  0x00007ff5c1544967 n/a (libnvidia-eglcore.so.545.29.06 + 0xd44967)
                #9  0x00007ff5c154732c n/a (libnvidia-eglcore.so.545.29.06 + 0xd4732c)
                #10 0x00007ff5c152c16c n/a (libnvidia-eglcore.so.545.29.06 + 0xd2c16c)
                #11 0x00007ff5c152c2d6 n/a (libnvidia-eglcore.so.545.29.06 + 0xd2c2d6)
                #12 0x00007ff5c1417405 n/a (libnvidia-eglcore.so.545.29.06 + 0xc17405)
                #13 0x0000563a929bdaa7 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0xdaaa7)
                #14 0x0000563a929bde17 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0xdae17)
                #15 0x0000563a929ba0e9 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0xd70e9)
                #16 0x0000563a929c9c32 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0xe6c32)
                #17 0x0000563a929c9bb8 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0xe6bb8)
                #18 0x0000563a929c9b12 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0xe6b12)
                #19 0x0000563a929c9aba n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0xe6aba)
                #20 0x0000563a929c9a82 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0xe6a82)
                #21 0x00007ff5c86e1943 execute_native_thread_routine (libstdc++.so.6 + 0xe1943)
                #22 0x00007ff5c84aa9eb n/a (libc.so.6 + 0x8c9eb)
                #23 0x00007ff5c852e7cc n/a (libc.so.6 + 0x1107cc)

                Stack trace of thread 57583:
                #0  0x00007ff5c852ebf6 epoll_wait (libc.so.6 + 0x110bf6)
                #1  0x0000563a9295d8ff n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0x7a8ff)
                #2  0x0000563a9295785a n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0x7485a)
                #3  0x0000563a929533bc n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0x703bc)
                #4  0x0000563a9296fde0 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0x8cde0)
                #5  0x0000563a9296fd50 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0x8cd50)
                #6  0x0000563a9296fc94 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0x8cc94)
                #7  0x0000563a9296fc08 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0x8cc08)
                #8  0x0000563a9296fa8a n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0x8ca8a)
                #9  0x00007ff5c86e1943 execute_native_thread_routine (libstdc++.so.6 + 0xe1943)
                #10 0x00007ff5c84aa9eb n/a (libc.so.6 + 0x8c9eb)
                #11 0x00007ff5c852e7cc n/a (libc.so.6 + 0x1107cc)

                Stack trace of thread 57585:
                #0  0x00007ff5c8520f6f __poll (libc.so.6 + 0x102f6f)
                #1  0x0000563a9298fc82 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0xacc82)
                #2  0x0000563a9296fe28 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0x8ce28)
                #3  0x0000563a9296fda3 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0x8cda3)
                #4  0x0000563a9296fcc0 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0x8ccc0)
                #5  0x0000563a9296fc24 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0x8cc24)
                #6  0x0000563a9296faaa n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0x8caaa)
                #7  0x00007ff5c86e1943 execute_native_thread_routine (libstdc++.so.6 + 0xe1943)
                #8  0x00007ff5c84aa9eb n/a (libc.so.6 + 0x8c9eb)
                #9  0x00007ff5c852e7cc n/a (libc.so.6 + 0x1107cc)

                Stack trace of thread 57589:
                #0  0x00007ff5c84a74ae n/a (libc.so.6 + 0x894ae)
                #1  0x00007ff5c84a9d40 pthread_cond_wait (libc.so.6 + 0x8bd40)
                #2  0x00007ff5c11eda0f n/a (libnvidia-eglcore.so.545.29.06 + 0x9eda0f)
                #3  0x00007ff5c151c818 n/a (libnvidia-eglcore.so.545.29.06 + 0xd1c818)
                #4  0x00007ff5c11f0dfa n/a (libnvidia-eglcore.so.545.29.06 + 0x9f0dfa)
                #5  0x00007ff5c84aa9eb n/a (libc.so.6 + 0x8c9eb)
                #6  0x00007ff5c852e7cc n/a (libc.so.6 + 0x1107cc)

                Stack trace of thread 57582:
                #0  0x00007ff5c91a558b n/a (ld-linux-x86-64.so.2 + 0x2158b)
                #1  0x00007ff5c9198c11 n/a (ld-linux-x86-64.so.2 + 0x14c11)
                #2  0x00007ff5c9185c89 n/a (ld-linux-x86-64.so.2 + 0x1c89)
                #3  0x00007ff5c91864da n/a (ld-linux-x86-64.so.2 + 0x24da)
                #4  0x00007ff5c91854e1 _dl_catch_exception (ld-linux-x86-64.so.2 + 0x14e1)
                #5  0x00007ff5c9185603 n/a (ld-linux-x86-64.so.2 + 0x1603)
                #6  0x00007ff5c84a64f7 n/a (libc.so.6 + 0x884f7)
                #7  0x00007ff5c84a6226 dlclose (libc.so.6 + 0x88226)
                #8  0x00007ff5c14d2a2b n/a (libnvidia-eglcore.so.545.29.06 + 0xcd2a2b)
                #9  0x00007ff5c122fd8a n/a (libnvidia-eglcore.so.545.29.06 + 0xa2fd8a)
                #10 0x00007ff5c12042d7 n/a (libnvidia-eglcore.so.545.29.06 + 0xa042d7)
                #11 0x00007ff5c2635bfa n/a (libEGL_nvidia.so.0 + 0x35bfa)
                #12 0x00007ff5c2635cb7 n/a (libEGL_nvidia.so.0 + 0x35cb7)
                #13 0x00007ff5c26a3015 n/a (libEGL_nvidia.so.0 + 0xa3015)
                #14 0x00007ff5c26a308d n/a (libEGL_nvidia.so.0 + 0xa308d)
                #15 0x00007ff5c2634fe9 n/a (libEGL_nvidia.so.0 + 0x34fe9)
                #16 0x00007ff5c26a32ef n/a (libEGL_nvidia.so.0 + 0xa32ef)
                #17 0x00007ff5c845ecc6 n/a (libc.so.6 + 0x40cc6)
                #18 0x00007ff5c845ee10 exit (libc.so.6 + 0x40e10)
                #19 0x00007ff5c8445cd7 n/a (libc.so.6 + 0x27cd7)
                #20 0x00007ff5c8445d8a __libc_start_main (libc.so.6 + 0x27d8a)
                #21 0x0000563a9290f295 n/a (/home/cylian91/test-gamescope/build/src/gamescope + 0x2c295)

I also look at the normal, unpatch version, look like its still the driver but they are way less thing:

                Stack trace of thread 57439:
                #0  0x00007f8fce0c8000 n/a (libnvidia-eglcore.so.545.29.06 + 0xcc8000)
                #1  0x00007f8fce0c9b90 n/a (libnvidia-eglcore.so.545.29.06 + 0xcc9b90)
                #2  0x00007f8fce0cae4b n/a (libnvidia-eglcore.so.545.29.06 + 0xccae4b)
                #3  0x00007f8fd4fe52a0 n/a (libvulkan.so.1 + 0x2d2a0)
                #4  0x00007f8fd4fdbbb4 n/a (libvulkan.so.1 + 0x23bb4)
                #5  0x00007f8fd4fdd5c5 n/a (libvulkan.so.1 + 0x255c5)
                #6  0x00007f8fd4ff261b vkCreateDevice (libvulkan.so.1 + 0x3a61b)
                #7  0x000055f5e7b43769 n/a (/home/cylian91/gamescope/build/src/gamescope + 0xd5769)
                #8  0x000055f5e7b423cd n/a (/home/cylian91/gamescope/build/src/gamescope + 0xd43cd)
                #9  0x000055f5e7b4e243 n/a (/home/cylian91/gamescope/build/src/gamescope + 0xe0243)
                #10 0x000055f5e7b02ebb n/a (/home/cylian91/gamescope/build/src/gamescope + 0x94ebb)
                #11 0x00007f8fd4845cd0 n/a (libc.so.6 + 0x27cd0)
                #12 0x00007f8fd4845d8a __libc_start_main (libc.so.6 + 0x27d8a)
                #13 0x000055f5e7a9a2b5 n/a (/home/cylian91/gamescope/build/src/gamescope + 0x2c2b5)

both crash are the same with ENABLE_GAMESCOPE_WSI=0

[cylian914]

look like this line is what cause the crash

[sharkautarch]

@cylian914 as for the issue you have with gamescope still crashing with the nvidia-fix branch, I just noticed that you had run gamescope with --prefer-vk-device 01:00.0 which is not quite the right gpu id format to pass to it... (I'm not sure how I didn't pick up on that until now...) I'm not sure whether or not it'll still crash anyways, but try running nvidia-fix gamescope like so: ./gamescope -e -w 1920 -h 900 -W 1600 -H 1080 --prefer-vk-device $(lspci -nn | grep -iE "[ ](vga|3D)" | tr ' ' $'\n' | grep 10de: | tr -d '[]') -- steam -tenfoot -steamos also make sure you have the nvidia-drm.modeset=1 kernel parameter set You may or may not also need to have nvidia_drm.fbdev=1 set

[KyunLFA]

@cylian914 How are you getting the segfault backtrace of embedded Gamescope? On my machine, running gdb or lldb on any graphical app ran straight from TTY such as embedded Gamescope causes the whole system to freeze, requiring me to use sysrq (system rescue) shortcut keys to get out of it... Did you do anything special, or is it just my install that is borked? I'm on CachyOS Linux

[cylian914]

@KyunLFA I use coredumpctl to retrieve the stack trace from the core dumb.