wlx-overlay-x segfaults immediately on DRM_IOCTL_PRIME_FD_TO_HANDLE - hybrid GPUs issue?

Thank you for the project and for your work!

When trying to run wlx-x, it segfaults immediately on the DRM_IOCTL_PRIME_FD_TO_HANDLE.

wlx-overlay-x dace93e Sun Sep 24 15:08:48 2023 monado 65d33502 Wed Sep 27 01:20:33 2023 monado-service env vars:

XRT_COMPOSITOR_COMPUTE=true XRT_COMPOSITOR_FORCE_VK_DISPLAY=0 ./monado-service

In this monado-service, i can run for example the hello_xr without issue:

XR_RUNTIME_JSON=/home/vanous/.local/share/openxr/1/openxr_monado.json hello_xr -g Vulkan

Hardware - a laptop with dual GPUs:

inxi -F: click to see log under this fold

``` inxi -F System: Host: vnkdell Kernel: 6.5.0-1-amd64 arch: x86_64 bits: 64 Desktop: GNOME v: 44.5 Distro: Debian GNU/Linux trixie/sid Machine: Type: Laptop System: Dell product: Inspiron 16 Plus 7620 v: N/A serial: Mobo: Dell model: 0V98G2 v: A00 serial: UEFI: Dell v: 1.17.0 date: 08/29/2023 Battery: ID-1: BAT0 charge: 84.3 Wh (100.0%) condition: 84.3/84.3 Wh (100.0%) CPU: Info: 14-core (6-mt/8-st) model: 12th Gen Intel Core i7-12700H bits: 64 type: MST AMCP cache: L2: 11.5 MiB Speed (MHz): avg: 413 min/max: 400/4600:4700:3500 cores: 1: 400 2: 400 3: 400 4: 400 5: 400 6: 400 7: 671 8: 400 9: 400 10: 400 11: 400 12: 400 13: 400 14: 400 15: 400 16: 400 17: 400 18: 400 19: 400 20: 400 Graphics: Device-1: Intel Alder Lake-P Integrated Graphics driver: i915 v: kernel Device-2: NVIDIA GA106M [GeForce RTX 3060 Mobile / Max-Q] driver: nvidia v: 525.125.06 Device-3: Microdia Integrated_Webcam_FHD driver: uvcvideo type: USB Display: wayland server: X.Org v: 1.23.2.1 with: Xwayland v: 23.2.1 compositor: gnome-shell driver: X: loaded: modesetting,nvidia unloaded: fbdev,nouveau,vesa dri: iris gpu: i915 resolution: 2048x1152~60Hz API: EGL v: 1.5 drivers: iris,nvidia,swrast platforms: gbm,wayland,x11,surfaceless,device API: OpenGL v: 4.6 vendor: intel mesa v: 23.2.1-1 renderer: Mesa Intel ```

Running wlx-x with these env vars:

__NV_PRIME_RENDER_OFFLOAD=1 VK_LAYER_NV_optimus=NVIDIA_only GLX_VENDOR_LIBRARY_NAME=nvidia XR_RUNTIME_JSON=/home/vanous/.local/share/openxr/1/openxr_monado.json ./wlx-overlay-x

before init
LOG in xrCreateInstance: Instance created
    createInfo->applicationInfo.applicationName: WlXrOverlay
    createInfo->applicationInfo.applicationVersion: 1
    createInfo->applicationInfo.engineName: StereoKit
    createInfo->applicationInfo.engineVersion: 12295
    appinfo.detected.engine.name: (null)
    appinfo.detected.engine.version: 0.0.0
    quirks.disable_vulkan_format_depth_stencil: false
LOG in xrCreateInstance: Selected devices
    Head: 'HTC Vive (vive)'
    Left: '<none>'
    Right: '<none>'
    Eyes: '<none>'
    Hand-Tracking Left: '<none>'
    Hand-Tracking Right: '<none>'
Segmentation fault

The segfault is: ioctl(6, DRM_IOCTL_PRIME_FD_TO_HANDLE, 0x7ffd5414ff2c) = -1 EINVAL (Invalid argument)

See full strace here: log.txt

OK, here is what i got:

Thread 1 "wlx-overlay-x" received signal SIGSEGV, Segmentation fault.
0x00007fffe037b941 in ?? () from /usr/lib/x86_64-linux-gnu/dri/iris_dri.so
(gdb) backtrace
#0  0x00007fffe037b941 in ?? () from /usr/lib/x86_64-linux-gnu/dri/iris_dri.so
#1  0x00007fffdf56d46f in ?? () from /usr/lib/x86_64-linux-gnu/dri/iris_dri.so
#2  0x00007fffdf55a278 in ?? () from /usr/lib/x86_64-linux-gnu/dri/iris_dri.so
#3  0x00007fffdf800a07 in ?? () from /usr/lib/x86_64-linux-gnu/dri/iris_dri.so
#4  0x00007fffdf8013e1 in ?? () from /usr/lib/x86_64-linux-gnu/dri/iris_dri.so
#5  0x00007ffff032ccc5 in client_gl_memobj_swapchain_create (xc=0x555556862c70, info=0x7fffffff9550, xscn=0x555556861970, out_cglsc=0x7fffffff94f8)
    at /home/vanous/bin/projects/xr/monado/src/xrt/compositor/client/comp_gl_memobj_swapchain.c:143
#6  0x00007ffff032c19f in client_gl_swapchain_create (xc=0x555556862c70, info=0x7fffffff9620, out_xsc=0x7fffffff9618)
    at /home/vanous/bin/projects/xr/monado/src/xrt/compositor/client/comp_gl_client.c:484
#7  0x00007ffff031e2cf in xrt_comp_create_swapchain (xc=0x555556862c70, info=0x7fffffff9620, out_xsc=0x7fffffff9618)
    at /home/vanous/bin/projects/xr/monado/src/xrt/include/xrt/xrt_compositor.h:1298
#8  0x00007ffff031ee0d in oxr_swapchain_common_create (log=0x7fffffff96c0, sess=0x5555567b9a80, createInfo=0x7fffffff9830, out_swapchain=0x7fffffff9680)
    at /home/vanous/bin/projects/xr/monado/src/xrt/state_trackers/oxr/oxr_swapchain.c:291
#9  0x00007ffff031f25f in oxr_swapchain_gl_create (log=0x7fffffff96c0, sess=0x5555567b9a80, createInfo=0x7fffffff9830, out_swapchain=0x7fffffff96d8)
    at /home/vanous/bin/projects/xr/monado/src/xrt/state_trackers/oxr/oxr_swapchain_gl.c:92
#10 0x00007ffff02f7cab in oxr_xrCreateSwapchain (session=0x5555567b9a80, createInfo=0x7fffffff9830, out_swapchain=0x7fffffff9828)
    at /home/vanous/bin/projects/xr/monado/src/xrt/state_trackers/oxr/oxr_api_swapchain.c:106
#11 0x00007ffff7aa7968 in xrCreateSwapchain () from /lib/x86_64-linux-gnu/libopenxr_loader.so.1
#12 0x000055555613bb42 in sk::openxr_create_swapchain (out_swapchain=..., type=XR_VIEW_CONFIGURATION_TYPE_PRIMARY_STEREO, color=true, array_size=2, format=35907, 
    width=1512, height=1680, sample_count=1)
    at /home/vanous/.cargo/registry/src/github.com-1ecc6299db9ec823/stereokit-sys-2.5.6/StereoKit/StereoKitC/xr_backends/openxr_view.cpp:527
#13 0x000055555613b326 in sk::openxr_update_swapchains (display=...)
    at /home/vanous/.cargo/registry/src/github.com-1ecc6299db9ec823/stereokit-sys-2.5.6/StereoKit/StereoKitC/xr_backends/openxr_view.cpp:422
#14 0x000055555613afe3 in sk::openxr_create_view (view_type=XR_VIEW_CONFIGURATION_TYPE_PRIMARY_STEREO, out_view=...)
    at /home/vanous/.cargo/registry/src/github.com-1ecc6299db9ec823/stereokit-sys-2.5.6/StereoKit/StereoKitC/xr_backends/openxr_view.cpp:375
#15 0x000055555613a9d1 in sk::openxr_views_create ()
    at /home/vanous/.cargo/registry/src/github.com-1ecc6299db9ec823/stereokit-sys-2.5.6/StereoKit/StereoKitC/xr_backends/openxr_view.cpp:266
#16 0x0000555556134577 in sk::openxr_init ()
    at /home/vanous/.cargo/registry/src/github.com-1ecc6299db9ec823/stereokit-sys-2.5.6/StereoKit/StereoKitC/xr_backends/openxr.cpp:534
#17 0x0000555556130681 in sk::platform_set_mode (mode=sk::display_type_stereo)
    at /home/vanous/.cargo/registry/src/github.com-1ecc6299db9ec823/stereokit-sys-2.5.6/StereoKit/StereoKitC/platforms/platform.cpp:159
#18 0x0000555556130531 in sk::platform_init ()
    at /home/vanous/.cargo/registry/src/github.com-1ecc6299db9ec823/stereokit-sys-2.5.6/StereoKit/StereoKitC/platforms/platform.cpp:69
#19 0x0000555556124571 in sk::systems_initialize ()
    at /home/vanous/.cargo/registry/src/github.com-1ecc6299db9ec823/stereokit-sys-2.5.6/StereoKit/StereoKitC/systems/system.cpp:138
#20 0x00005555560d1878 in sk::sk_init (settings=...)
    at /home/vanous/.cargo/registry/src/github.com-1ecc6299db9ec823/stereokit-sys-2.5.6/StereoKit/StereoKitC/stereokit.cpp:320
#21 0x00005555560bef66 in stereokit::Settings::init (self=...) at src/lib.rs:616
#22 0x00005555556e3a56 in wlx_overlay_x::main () at src/main.rs:126
#23 0x00005555556cf78b in core::ops::function::FnOnce::call_once<fn(), ()> () at /usr/src/rustc-1.70.0/library/core/src/ops/function.rs:250
#24 0x000055555573f35e in std::sys_common::backtrace::__rust_begin_short_backtrace<fn(), ()> (f=0x5555556e3860 <wlx_overlay_x::main>)
    at /usr/src/rustc-1.70.0/library/std/src/sys_common/backtrace.rs:134
#25 0x0000555555794ab1 in std::rt::lang_start::{closure#0}<()> () at /usr/src/rustc-1.70.0/library/std/src/rt.rs:166
#26 0x000055555616be81 in core::ops::function::impls::{impl#2}::call_once<(), (dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe)> (self=..., args=<optimized out>) at library/core/src/ops/function.rs:287
#27 std::panicking::try::do_call<&(dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe), i32> (data=<optimized out>)
--Type <RET> for more, q to quit, c to continue without paging--
    at library/std/src/panicking.rs:485
#28 std::panicking::try<i32, &(dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe)> (f=...)
    at library/std/src/panicking.rs:449
#29 std::panic::catch_unwind<&(dyn core::ops::function::Fn<(), Output=i32> + core::marker::Sync + core::panic::unwind_safe::RefUnwindSafe), i32> (f=...)
    at library/std/src/panic.rs:140
#30 std::rt::lang_start_internal::{closure#2} () at library/std/src/rt.rs:148
#31 std::panicking::try::do_call<std::rt::lang_start_internal::{closure_env#2}, isize> (data=<optimized out>) at library/std/src/panicking.rs:485
#32 std::panicking::try<isize, std::rt::lang_start_internal::{closure_env#2}> (f=...) at library/std/src/panicking.rs:449
#33 0x000055555617f14b in std::panic::catch_unwind<std::rt::lang_start_internal::{closure_env#2}, isize> (f=...) at library/std/src/panic.rs:140
#34 std::rt::lang_start_internal (main=..., argc=<optimized out>, argv=<optimized out>, sigpipe=<optimized out>) at library/std/src/rt.rs:148
#35 0x0000555555794a8a in std::rt::lang_start<()> (main=0x5555556e3860 <wlx_overlay_x::main>, argc=1, argv=0x7fffffffdbd8, sigpipe=0)
    at /usr/src/rustc-1.70.0/library/std/src/rt.rs:165
#36 0x00005555556e45be in main ()
#37 0x00007ffff759d6ca in __libc_start_call_main (main=main@entry=0x5555556e45a0 <main>, argc=argc@entry=1, argv=argv@entry=0x7fffffffdbd8)
    at ../sysdeps/nptl/libc_start_call_main.h:58
#38 0x00007ffff759d785 in __libc_start_main_impl (main=0x5555556e45a0 <main>, argc=1, argv=0x7fffffffdbd8, init=<optimized out>, fini=<optimized out>, 
    rtld_fini=<optimized out>, stack_end=0x7fffffffdbc8) at ../csu/libc-start.c:360
#39 0x000055555564d311 in _start ()

galister / wlx-overlay-x

wlx-overlay-x segfaults immediately on DRM_IOCTL_PRIME_FD_TO_HANDLE - hybrid GPUs issue? #5