search

ValveSoftware / gamescope

SteamOS session compositing window manager
Other
2.84k stars 189 forks source link

WSI layer failures with non null oldSwapchain #1091

Open nowrep opened 6 months ago

nowrep commented 6 months ago

After a2f8db9fdb20975582535828533619603705b257 WSI layer will deadlock after re-creating swapchain. Can be reproduced on both Deck (SteamOS preview) and desktop radv.

# hangs when starting to play second file
mpv --vo=gpu-next --gpu-api=vulkan sdr-video.mkv hdr-video.mkv

This is how the swapchain is created: https://code.videolan.org/videolan/libplacebo/-/blob/52314e0e435fbcb731e326815d4091ed0ba27475/src/vulkan/swapchain.c#L574

Before a2f8db9fdb20975582535828533619603705b257 there is still a rare chance to get VK_ERROR_INITIALIZATION_FAILED from vkCreateSwapchainKHR (I think this was because the surface was still being used by old swapchain).

Only happens when application sets VkSwapchainCreateInfoKHR::oldSwapchain.

Joshua-Ashton commented 6 months ago

Thanks, I think we need to NULL that out when going between surfaces potentially.

Do you have a full log? Would be interesting to see if you are getting flip constantly

Joshua-Ashton commented 6 months ago

(That is to say, if you are continuously getting wayland surface, then this may be a Mesa bug)

Joshua-Ashton commented 6 months ago

Ohhh we should probably destroy the content override for oldSwapchain too at create time!

nowrep commented 6 months ago

Log:

[Gamescope WSI] Creating Gamescope surface: xid: 0x400002
[Gamescope WSI] Atom of T was wrong type. Expected XCB_ATOM_CARDINAL.
wlserver: [types/wlr_compositor.c:692] New wlr_surface 0x5573179d8b10 (res 0x557317af04e0)[Gamescope WSI] Made gamescope surface for xid: 0x400002

[Gamescope WSI] Surface state:
  steam app id:                  0
  window xid:                    0x400002
  wayland surface res id:        5
  layer client flags:            0x0
  server hdr output enabled:     false
  hdr formats exposed to client: false
wlserver: [types/wlr_compositor.c:692] New wlr_surface 0x5573179d7c60 (res 0x557317af5160)
[Gamescope WSI] Creating swapchain for xid: 0x400002 - minImageCount: 4 - format: VK_FORMAT_R16G16B16A16_UNORM - colorspace: VK_COLOR_SPACE_SRGB_NONLINEAR_KHR - flip: true
xwm: Unhandled client message: _NET_REQUEST_FRAME_EXTENTS
[Gamescope WSI] Created swapchain for xid: 0x400002 - imageCount: 4
[Gamescope WSI] Swapchain recieved new refresh cycle: 8.33ms
vulkan: Creating Gamescope nested swapchain with format 64 and colorspace 0
pipewire: renegotiating stream params (size: 1135x1388)
vulkan: Creating Gamescope nested swapchain with format 64 and colorspace 0
pipewire: renegotiating stream params (size: 1706x1388)
[Gamescope WSI] Creating swapchain for xid: 0x400002 - minImageCount: 4 - format: VK_FORMAT_R16G16B16A16_UNORM - colorspace: VK_COLOR_SPACE_SRGB_NONLINEAR_KHR - flip: true
[Gamescope WSI] Created swapchain for xid: 0x400002 - imageCount: 4
wlserver: create_swapchain: Surface already had a gamescope_swapchain! Overriding.
xwm: waylandres but no win

Backtrace:

Thread 13 (Thread 0x7f7b267fe6c0 (LWP 225201) "vo"):
#0  0x00007f7b6013af6f in __GI___poll (fds=fds@entry=0x7f7b267fd090, nfds=nfds@entry=1, timeout=timeout@entry=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
#1  0x00007f7b604d2b5e in poll (__timeout=-1, __nfds=1, __fds=0x7f7b267fd090) at /usr/include/bits/poll2.h:39
#2  wl_display_poll (display=0x7f7b1008fca0, events=1) at ../wayland-1.22.0/src/wayland-client.c:1914
#3  wl_display_dispatch_queue (queue=<optimized out>, display=<optimized out>) at ../wayland-1.22.0/src/wayland-client.c:1987
#4  wl_display_dispatch_queue (display=0x7f7b1008fca0, queue=0x7f7b10885260) at ../wayland-1.22.0/src/wayland-client.c:1960
#5  0x00007f7b071a0cd2 in  () at /usr/lib/libvulkan_radeon.so
#6  0x00007f7b0719b91b in  () at /usr/lib/libvulkan_radeon.so
#7  0x00007f7b442d4ed8 in  () at /usr/lib/libVkLayer_FROG_gamescope_wsi_x86_64.so
#8  0x00007f7b62398239 in vk_sw_submit_frame (sw=0x7f7b10152810) at ../src/vulkan/swapchain.c:828 => vkQueuePresentKHR()
#9  0x00005616cceea0e0 in flip_page (vo=0x5616ce81c100) at ../mpv/video/out/vo_gpu_next.c:1106
#10 0x00005616ccee4ff9 in render_frame (vo=0x5616ce81c100) at ../mpv/video/out/vo.c:949
#11 vo_thread (ptr=<optimized out>) at ../mpv/video/out/vo.c:1074
#12 0x00007f7b600c49eb in start_thread (arg=<optimized out>) at pthread_create.c:444
#13 0x00007f7b601487cc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78
Joshua-Ashton commented 6 months ago

I'll send a branch tomorrow :)

cgutman commented 4 months ago

Moonlight is also affected by this regression, which has now made it to SteamOS Stable branch with 3.5.17.

You can also use libplacebo's test binary directly to reproduce:

  1. git clone --recursive libplacebo from https://github.com/haasn/libplacebo and compile it with meson (ensuring vulkan is enabled)
  2. Download a sample video to play, such as this one from https://kodi.wiki/view/Samples
  3. Run plplay <video file> inside Gamescope
  4. Click one of the options in the plplay overlay UI that causes the swapchain to be recreated (I chose the Fullscreen checkbox)
sharkautarch commented 4 months ago

what happens if you try running gamescope with: GAMESCOPE_WSI_FORCE_BYPASS=1 gamescope ...

nowrep commented 4 months ago

This happens also when there is one single window with no sub windows/surfaces, so bypass doesn't make a difference.