Issues with the WSI layer, hdr, and steam overlay / input failing to hook

[VoidAndCaffeine]

I run gamescope more or less a a daemon to get around #835. rather than directly launching games with gamescope (vars) -- %command%, I use gamemode to automatically start gamescope and then pass DISPLAY=:2 DXVK_HDR=1to the game. I decided i was going to give plasma6 + hdr a go but I found that my (admittedly niche) setup is broken. Launching the normal way, gamescope (vars) -- %command%, properly initiates hdr.

To reproduce: (my example will be cyberpunk as I don't own ori)

1. make sure gamescope isn't already running pkill gamescope
2. in your term run gamescope -e -f -h 1440 -r 165 --hdr-enabled specifically the git master version (no others work on my system)
3. from steam edit the launch options of cp77 to be DISPLAY=:2 DXVK_HDR=1 %command% --launcher-skip
4. launch cp77
5. the game will launch and output hdr but gamescope will display it as if it is sdr causing the image to be over saturated and over contrasty.

here is the output from the above example:

``` No CAP_SYS_NICE, falling back to regular-priority compute and threads. Performance will be affected. ATTENTION: default value of option vk_khr_present_wait overridden by environment. vulkan: selecting physical device 'AMD Radeon RX 7800 XT (RADV NAVI32)': queue family 1 (general queue family 0) vulkan: physical device supports DRM format modifiers wlserver: [backend/headless/backend.c:67] Creating headless backend xdg_backend: Seat name: vulkan: supported DRM formats for sampling usage: vulkan: AR24 (0x34325241) vulkan: XR24 (0x34325258) vulkan: AB24 (0x34324241) vulkan: XB24 (0x34324258) vulkan: RG16 (0x36314752) vulkan: NV12 (0x3231564E) vulkan: AB4H (0x48344241) vulkan: XB4H (0x48344258) vulkan: AB48 (0x38344241) vulkan: XB48 (0x38344258) vulkan: AB30 (0x30334241) vulkan: XB30 (0x30334258) vulkan: AR30 (0x30335241) vulkan: XR30 (0x30335258) wlserver: Running compositor on wayland display 'gamescope-0' wlserver: [backend/headless/backend.c:17] Starting headless backend wlserver: [xwayland/sockets.c:64] Failed to bind socket @/tmp/.X11-unix/X0: Address already in use wlserver: [xwayland/server.c:107] Starting Xwayland on :2 The XKEYBOARD keymap compiler (xkbcomp) reports: > Warning: Could not resolve keysym XF86CameraAccessEnable > Warning: Could not resolve keysym XF86CameraAccessDisable > Warning: Could not resolve keysym XF86CameraAccessToggle > Warning: Could not resolve keysym XF86NextElement > Warning: Could not resolve keysym XF86PreviousElement > Warning: Could not resolve keysym XF86AutopilotEngageToggle > Warning: Could not resolve keysym XF86MarkWaypoint > Warning: Could not resolve keysym XF86Sos > Warning: Could not resolve keysym XF86NavChart > Warning: Could not resolve keysym XF86FishingChart > Warning: Could not resolve keysym XF86SingleRangeRadar > Warning: Could not resolve keysym XF86DualRangeRadar > Warning: Could not resolve keysym XF86RadarOverlay > Warning: Could not resolve keysym XF86TraditionalSonar > Warning: Could not resolve keysym XF86ClearvuSonar > Warning: Could not resolve keysym XF86SidevuSonar > Warning: Could not resolve keysym XF86NavInfo Errors from xkbcomp are not fatal to the X server wlserver: [types/wlr_compositor.c:771] New wlr_surface 0x5d9740711b50 (res 0x5d97407123b0) wlserver: [xwayland/server.c:272] Xserver is ready pipewire: stream state changed: connecting pipewire: stream state changed: paused pipewire: stream available on node ID: 108 xwm: Embedded, no cursor set. Using left_ptr by default. vblank: Using timerfd. xdg_backend: PreferredMetadata: Red: 0.708 0.292, Green: 0.17 0.797, Blue: 0.131 0.046, White: 0.3127 0.329, Max Luminance: 409 nits, Min Luminance: 0 nits, Max Full Frame Luminance: 351 nits josh edid: Patching res 800x1280 -> 2560x1440 pipewire: renegotiating stream params (size: 1280x720) pipewire: renegotiating stream params (size: 1440x2560) wlserver: [types/wlr_compositor.c:771] New wlr_surface 0x5d97406f1b50 (res 0x5d9740628d30) xwm: got the same buffer committed twice, ignoring. The XKEYBOARD keymap compiler (xkbcomp) reports: > Warning: Unsupported maximum keycode 708, clipping. > X11 cannot support keycodes above 255. > Warning: Could not resolve keysym XF86CameraAccessEnable > Warning: Could not resolve keysym XF86CameraAccessDisable > Warning: Could not resolve keysym XF86CameraAccessToggle > Warning: Could not resolve keysym XF86NextElement > Warning: Could not resolve keysym XF86PreviousElement > Warning: Could not resolve keysym XF86AutopilotEngageToggle > Warning: Could not resolve keysym XF86MarkWaypoint > Warning: Could not resolve keysym XF86Sos > Warning: Could not resolve keysym XF86NavChart > Warning: Could not resolve keysym XF86FishingChart > Warning: Could not resolve keysym XF86SingleRangeRadar > Warning: Could not resolve keysym XF86DualRangeRadar > Warning: Could not resolve keysym XF86RadarOverlay > Warning: Could not resolve keysym XF86TraditionalSonar > Warning: Could not resolve keysym XF86ClearvuSonar > Warning: Could not resolve keysym XF86SidevuSonar > Warning: Could not resolve keysym XF86NavInfo Errors from xkbcomp are not fatal to the X server xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ wlserver: [types/wlr_compositor.c:771] New wlr_surface 0x5d97406f1540 (res 0x5d974063f2c0) xwm: Rejecting WM_CHANGE_STATE to ICONIC for window 0x1a00001 wlserver: [types/wlr_compositor.c:771] New wlr_surface 0x5d97403d3dd0 (res 0x5d974063fce0) xwm: waylandres but no win xwm: waylandres but no win xwm: error 3: BadWindow (invalid Window parameter) request 15 minor 0 serial 14150 gamescope: Received Terminated signal, attempting shutdown! (EE) failed to read Wayland events: Broken pipe ```

and here is the output launching the normal way:

``` No CAP_SYS_NICE, falling back to regular-priority compute and threads. Performance will be affected. ATTENTION: default value of option vk_x11_override_min_image_count overridden by environment. ATTENTION: default value of option vk_khr_present_wait overridden by environment. vulkan: selecting physical device 'AMD Radeon RX 7800 XT (RADV NAVI32)': queue family 1 (general queue family 0) vulkan: physical device supports DRM format modifiers wlserver: [backend/headless/backend.c:67] Creating headless backend xdg_backend: Seat name: vulkan: supported DRM formats for sampling usage: vulkan: AR24 (0x34325241) vulkan: XR24 (0x34325258) vulkan: AB24 (0x34324241) vulkan: XB24 (0x34324258) vulkan: RG16 (0x36314752) vulkan: NV12 (0x3231564E) vulkan: AB4H (0x48344241) vulkan: XB4H (0x48344258) vulkan: AB48 (0x38344241) vulkan: XB48 (0x38344258) vulkan: AB30 (0x30334241) vulkan: XB30 (0x30334258) vulkan: AR30 (0x30335241) vulkan: XR30 (0x30335258) wlserver: Running compositor on wayland display 'gamescope-0' wlserver: [backend/headless/backend.c:17] Starting headless backend wlserver: [xwayland/sockets.c:64] Failed to bind socket @/tmp/.X11-unix/X0: Address already in use wlserver: [xwayland/server.c:107] Starting Xwayland on :2 The XKEYBOARD keymap compiler (xkbcomp) reports: > Warning: Could not resolve keysym XF86CameraAccessEnable > Warning: Could not resolve keysym XF86CameraAccessDisable > Warning: Could not resolve keysym XF86CameraAccessToggle > Warning: Could not resolve keysym XF86NextElement > Warning: Could not resolve keysym XF86PreviousElement > Warning: Could not resolve keysym XF86AutopilotEngageToggle > Warning: Could not resolve keysym XF86MarkWaypoint > Warning: Could not resolve keysym XF86Sos > Warning: Could not resolve keysym XF86NavChart > Warning: Could not resolve keysym XF86FishingChart > Warning: Could not resolve keysym XF86SingleRangeRadar > Warning: Could not resolve keysym XF86DualRangeRadar > Warning: Could not resolve keysym XF86RadarOverlay > Warning: Could not resolve keysym XF86TraditionalSonar > Warning: Could not resolve keysym XF86ClearvuSonar > Warning: Could not resolve keysym XF86SidevuSonar > Warning: Could not resolve keysym XF86NavInfo Errors from xkbcomp are not fatal to the X server wlserver: [types/wlr_compositor.c:771] New wlr_surface 0x58e899b1b620 (res 0x58e899b162a0) wlserver: [xwayland/server.c:272] Xserver is ready pipewire: stream state changed: connecting pipewire: stream state changed: paused pipewire: stream available on node ID: 109 xwm: Embedded, no cursor set. Using left_ptr by default. vblank: Using timerfd. xdg_backend: PreferredMetadata: Red: 0.708 0.292, Green: 0.17 0.797, Blue: 0.131 0.046, White: 0.3127 0.329, Max Luminance: 409 nits, Min Luminance: 0 nits, Max Full Frame Luminance: 351 nits josh edid: Patching res 800x1280 -> 2560x1440 pipewire: renegotiating stream params (size: 1280x720) pipewire: renegotiating stream params (size: 2560x1440) fsync: up and running. ATTENTION: default value of option vk_x11_override_min_image_count overridden by environment. ATTENTION: default value of option vk_khr_present_wait overridden by environment. ATTENTION: default value of option vk_x11_override_min_image_count overridden by environment. ATTENTION: default value of option vk_khr_present_wait overridden by environment. Setting breakpad minidump AppID = 1091500 Steam_SetMinidumpSteamID: Caching Steam ID: 76561198301253563 [API loaded no] ATTENTION: default value of option vk_x11_override_min_image_count overridden by environment. ATTENTION: default value of option vk_khr_present_wait overridden by environment. wine: setpriority -2 for pid -1 failed: 3 ATTENTION: default value of option vk_x11_override_min_image_count overridden by environment. ATTENTION: default value of option vk_khr_present_wait overridden by environment. wine: setpriority -2 for pid -1 failed: 3 ATTENTION: default value of option vk_x11_override_min_image_count overridden by environment. ATTENTION: default value of option vk_khr_present_wait overridden by environment. ATTENTION: default value of option vk_x11_override_min_image_count overridden by environment. ATTENTION: default value of option vk_khr_present_wait overridden by environment. [Gamescope WSI] Forcing on VK_EXT_swapchain_maintenance1. ATTENTION: default value of option vk_x11_override_min_image_count overridden by environment. ATTENTION: default value of option vk_khr_present_wait overridden by environment. ATTENTION: default value of option vk_x11_override_min_image_count overridden by environment. ATTENTION: default value of option vk_khr_present_wait overridden by environment. [Gamescope WSI] Forcing on VK_EXT_swapchain_maintenance1. wine: setpriority -19 for pid -1 failed: 3 wine: setpriority -19 for pid -1 failed: 3 wlserver: [types/wlr_compositor.c:771] New wlr_surface 0x58e899abda10 (res 0x58e8996d4440) xwm: got the same buffer committed twice, ignoring. The XKEYBOARD keymap compiler (xkbcomp) reports: > Warning: Unsupported maximum keycode 708, clipping. > X11 cannot support keycodes above 255. > Warning: Could not resolve keysym XF86CameraAccessEnable > Warning: Could not resolve keysym XF86CameraAccessDisable > Warning: Could not resolve keysym XF86CameraAccessToggle > Warning: Could not resolve keysym XF86NextElement > Warning: Could not resolve keysym XF86PreviousElement > Warning: Could not resolve keysym XF86AutopilotEngageToggle > Warning: Could not resolve keysym XF86MarkWaypoint > Warning: Could not resolve keysym XF86Sos > Warning: Could not resolve keysym XF86NavChart > Warning: Could not resolve keysym XF86FishingChart > Warning: Could not resolve keysym XF86SingleRangeRadar > Warning: Could not resolve keysym XF86DualRangeRadar > Warning: Could not resolve keysym XF86RadarOverlay > Warning: Could not resolve keysym XF86TraditionalSonar > Warning: Could not resolve keysym XF86ClearvuSonar > Warning: Could not resolve keysym XF86SidevuSonar > Warning: Could not resolve keysym XF86NavInfo Errors from xkbcomp are not fatal to the X server xdg_backend: Changed refresh to: 144.001hz [Gamescope WSI] Creating Gamescope surface: xid: 0x180003b [Gamescope WSI] Atom of T was wrong type. Expected XCB_ATOM_CARDINAL. wlserver: [types/wlr_compositor.c:771] New wlr_surface 0x58e899abe4a0 (res 0x58e8996d5c30)[Gamescope WSI] Made gamescope surface for xid: 0x180003b [Gamescope WSI] Surface state: steam app id: 1091500 window xid: 0x180003b wayland surface res id: 5 layer client flags: 0x4 server hdr output enabled: true hdr formats exposed to client: true xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ xwm: Unhandled NET_WM_STATE property change: _KDE_NET_WM_STATE_SKIP_SWITCHER xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_ABOVE xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_VERT xwm: Unhandled NET_WM_STATE property change: _NET_WM_STATE_MAXIMIZED_HORZ wine: setpriority -2 for pid -1 failed: 3 [Gamescope WSI] minImageCount overridden by vk_x11_override_min_image_count: 4 [Gamescope WSI] Creating swapchain for xid: 0x180003b - minImageCount: 4 - format: VK_FORMAT_R16G16B16A16_SFLOAT - colorspace: VK_COLOR_SPACE_EXTENDED_SRGB_LINEAR_EXT - flip: true [Gamescope WSI] Created swapchain for xid: 0x180003b - imageCount: 4 [Gamescope WSI] VkHdrMetadataEXT: display primaries: r: 0.64 0.33 g: 0.3 0.6 b: 0.15 0.06 w: 0.3127 0.329 mastering luminance: min 0.1 nits, max 500 nits maxContentLightLevel: 500 nits maxFrameAverageLightLevel: 100 nits [Gamescope WSI] Creating swapchain for xid: 0x180003b - minImageCount: 4 - format: VK_FORMAT_R16G16B16A16_SFLOAT - colorspace: VK_COLOR_SPACE_EXTENDED_SRGB_LINEAR_EXT - flip: true [Gamescope WSI] Created swapchain for xid: 0x180003b - imageCount: 4 [Gamescope WSI] VkHdrMetadataEXT: display primaries: r: 0.64 0.33 g: 0.3 0.6 b: 0.15 0.06 w: 0.3127 0.329 mastering luminance: min 0.1 nits, max 500 nits maxContentLightLevel: 500 nits maxFrameAverageLightLevel: 100 nits [Gamescope WSI] Swapchain recieved new refresh cycle: 6.06ms wlserver: [types/wlr_compositor.c:771] New wlr_surface 0x58e899b62050 (res 0x58e899b61ef0) xwm: Rejecting WM_CHANGE_STATE to ICONIC for window 0x1a00001 wlserver: [types/wlr_compositor.c:771] New wlr_surface 0x58e899b62420 (res 0x58e899b627f0) xwm: waylandres but no win xwm: waylandres but no win xwm: error 3: BadWindow (invalid Window parameter) request 15 minor 0 serial 15590 gamescope: children shut down! (EE) failed to read Wayland events: Broken pipe ```

[Joshua-Ashton]

You can't do HDR on X11/XWayland. You need to set WAYLAND_DISPLAY

[Joshua-Ashton]

Sorry, you are launching the game, so it needs GAMESCOPE_WAYLAND_DISPLAY, not WAYLAND_DISPLAY.

It also needs ENABLE_GAMESCOPE_WSI=1

I don't get why this would change anything about overlay though. Does it work in this way?

[VoidAndCaffeine]

I believe ENABLE_GAMESCOPE_WSI=1 is set in my environment but I will double check. I will also give GAMESCOPE_WAYLAND_DISPLAY & WAYLAND_DISPLAY a try.

do I need to run gamescope with --expose-wayland?

sence the big Wayland update, steam overlay doesn't attach to gamescope when launching the normal way, afik it is an issue with the overlay not gamescope. when the game connects to gamescope via DISPLAY the overlay attaches to gamescope just fine. I got the workaround form the discussion at #835.

I will be at my box to test in a few hours.

[Joshua-Ashton]

You only need GAMESCOPE_WAYLAND_DISPLAY, no extra flag

[VoidAndCaffeine]

ok so runing a game with gamescope run from a terminal with the launch options DXVK_HDR=1 GAMESCOPE_WAYLAND_DISPLAY=gamescope-0 %command% --launcher-skip does nothing, the game lauches as a normal window outside of gamescope.

running a game with the same setup and the lauch options DXVK_HDR=1 WAYLAND_DISPLAY=gamescope-0 %command% --launcher-skip has the same result.

the same applies when gamescope is launched with --expose-wayland.

From what I can tell I assume this is expected as proton/wine does not support native Wayland so Wayland environment variables are ignored. Your comments @Joshua-Ashton lead me to assume there are things that gamescope does when launching an application directly via gamescope (vars) -- that it doesn't do when a client connects via DISPLAY=:n. is that correct?

Assuming that is correct and intentional than this issue can be closed as won't fix.

Out of curiosity, and again assuming the above is true, how difficult would this be: an additional launch option --existing when there are no other gamescope instances running it operates like normal and launches a new gamescope instance. however when there is an existing instance of gamescope it does the necessary "hooks" to a new program but connects it to the existing gamescope. In addition to fixing my (again, niche) issue, working around steam overlay's current incompatibility with nested gamescope on wayland. It would also remove the need to have different launch options between desktop and chimera's gamescope session. I know yall are busy so this is more of a though experiment than a feature request :)

[Joshua-Ashton]

I tried removing the code in spawn_client to mess with LD_PRELOAD which got the overlay to work and hook into the game inside Gamescope -- but unfortunately it ends up breaking all input once the overlay comes up. So /shrug

[dklementowski]

I can sucessfuly run GoW with HDR on Plasma 6 using Gamescope, but no way to get the overlay working makes it unusable, because no proper controller mapping (my desktop layout persists even though game is running). What's positive, is that it doesn't break VRR like it does in embedded mode (which I only forced to run with amd-color patched kernel and gamescope with patches to use it, but is highly unstable and part of games have broken colors no matter if SDR or HDR). Related issue: https://github.com/ValveSoftware/gamescope/issues/1006

A side note, it would be cool to have the ability to run Gamescope with all the overlay and input handled inside like it is done in SteamOS embedded session. That would solve both this particular problem, but also overlay being broken for Flatpak apps https://github.com/flatpak/flatpak/issues/1756 (which straight-up cancels Steam Input for all Flatpak launchers, emulators, EmuDeck, all that fun stuff). But of course I don't know how the internals work and if it's even feasible.

[VoidAndCaffeine]

I tried removing the code in spawn_client to mess with LD_PRELOAD which got the overlay to work and hook into the game inside Gamescope -- but unfortunately it ends up breaking all input once the overlay comes up. So /shrug

I duplicated what I assume you did @Joshua-Ashton, I removed the if ( strstr( pchPreloadCopy + i, "gameoverlayrenderer.so" ) == nullptr ) check in spawn_client. Then running as I usually do with /home/void/git-clones/gamescope/build/src/gamescope -e -f -h 1440 -r 165 --hdr-enabled -- gamemoderun %command% --launcher-skip resulted in the overlay working alongside hdr, native input with my dualsence and steaminput. It does appear that the overlay is hooking into the game's window rather than gamescope (I don't know if it usually does or not.).

I assume gameoverlayrenderer.so is being removed from LD_PRELOAD so that the overlay can be attached elsewhere in gamescope and that reattach isn't happening, is that correct? If so where might I look for that reattach so that I might f around with it.

as a side note for some reason running cyberpunk with --hdr-debug-heatmap causes a crash a couple seconds in on both git master and v3.14.2

[VoidAndCaffeine]

Here is something interesting installing the git master with my pr and then lauching results in a working overlay, but as soon as I run sudo setcap 'CAP_SYS_NICE=eip' $(which gamescope) the overlay fails to hook. This explains why the issue first occurred when I switched to gentoo as CAP_SYS_NICE is set as part of the ebuild.

EDIT: Although there is still some other issue as installing a clean git master still results in no overlay