Erroneous autodiscovery of nested Plasma session in non-Steam applications (no WAYLAND_DISPLAY specified)

[zetaPRIME]

Your system information

• Steam client version: Jan 5 beta
• SteamOS version: 3.4.5 (preview channel)
• Opted into Steam client beta?: Yes
• Opted into SteamOS beta?: Yes
• Have you checked for updates in Settings > System?: Yes

Please describe your issue in as much detail as possible:

Expectation: Applications launched from within Steam attach to gamescope as normal and thus display under their application within Game Mode.
Reality: If a nested Plasma session is running, non-Steam (and only non-Steam) applications attach to the Plasma session instead of gamescope.

Steps for reproducing this issue:

1. Launch a Plasma session within gamescope as described here https://www.reddit.com/r/SteamDeck/comments/zqgx9g/ (it doesn't matter if the version with the separate dbus session is used or not)
2. Attempt to launch a non-Steam title (seen at least with flatpak versions of Xonotic and Sonic Robo Blast 2 Kart); observe as it attaches to your nested Plasma session instead of gamescope
3. Edit your non-steam game entry (if flatpak) to include --nosocket=wayland and relaunch; it should attach to gamescope as expected this time.

Suggested fix:

Set the WAYLAND_DISPLAY environment variable by default when launching a non-Steam title, either to a value valid for gamescope, or to an explicitly invalid value (such as no) to prevent the host application from automatically trying wayland-0 and getting slurped into Plasma as a result.

[bertogg]

If I'm not wrong apps are launched with DISPLAY set, WAYLAND_DISPLAY unset and XDG_SESSION_TYPE=x11, so they should try to use X11, and some of them do (try OBS Studio for example, see also #842).

Ideally apps should honor the XDG_SESSION_TYPE variable.

Some possible workarounds:

• Set WAYLAND_DISPLAY to a bogus value as you say.
• Use a different socket so it's not discovered automatically. kwin_wayland does have the --socket option, maybe you can tweak your script to do that.

[zetaPRIME]

I tried to look into that second one, but I couldn't determine if there was a way to make kwin_wayland_wrapper specifically use a particular socket name.

[bertogg]

Maybe it's easy to do manually what the wrapper does?

https://github.com/KDE/kwin/blob/master/src/helpers/wayland_wrapper/kwin_wrapper.cpp