search

ValveSoftware / steam-for-linux

Issue tracking for the Steam for Linux beta client
4.21k stars 175 forks source link

Gnome/Wayland asks to Allow Remote Interaction to use Xbox Controller as mouse #10442

Open aliasbody opened 7 months ago

aliasbody commented 7 months ago

Your system information

Please describe your issue in as much detail as possible:

When using Gnome with Wayland with a Xbox One X Controller, and when pressing the Guide button + any other option (like joystick up) instead of starting controlling the desktop there is a popup that appears asking to "Allow remote interactions".

image

Allowing it makes the controller work as a mouse (control volume etc..), but once Steam is closed or the controller is disconnected/reconnected, if we try the same operation it asks for the same permissions again.

Going to "Settings > Controller > Desktop Layout (EDIT) > Disable Steam Input" doesn't solve the problem, it still shows the popup.

Steps for reproducing this issue:

  1. Connect a Xbox One X Controller via Bluetooth
  2. Press Guide + "Joystick Up"
makergamer commented 5 months ago

Could this be part of the issue: Xwayland should have the XDG portal disabled by default with a Wayland opt-in command line option to enable it. It's best to read the last URL below about the merge request to see what's going on.

Gear:

More details for the Steam for Linux on Fedora Silverblue 39 with Gnome 45, combinations tested: With Steam (flatpak or rpm-ostree) closed -- If closed, the Remote Desktop dialog doesn't appear when the following controllers are used in any fashion.

Steam only needs to be open for the symptom.

My research findings: Trying to find the true root cause so I can enjoy my Steam Library when I've my Framework 13 only, I finally came across these posts. These are "breadcrumbs" in the order I found them:

So if I understand it correctly, it looks like a portion of Wayland required an alteration to not default to being "always responsive" to XTEST. As the merge request was just 5 months ago (as of today), I've no clue as to how soon the fix will propagate into the next release(s) of Gnome and Fedora.

If that merge is truly the fix, it might behoove Valve to have one of their advocates reach out to Gnome and help get this fix fast tracked. This "bug" is impacting all Steam for Linux users who use Wayland -- particularly with Gnome combinations.

Thanks. Cheers.

quidamphx commented 4 months ago

You found more on this topic than I had until I stumbled here and read your links about it. Thanks for adding all that detail, it gives me more to look into for the sake of understanding the issue.

It's great from an accessibility perspective to be able to have Steam auto-start on system boot and use the controller to navigate without an additional prompt being needed. It's one feature I didn't realise I used in Windows as much as I did until I couldn't do the same in Fedora.

bendavis78 commented 3 months ago

It's great from an accessibility perspective to be able to have Steam auto-start on system boot and use the controller to navigate without an additional prompt being needed.

Does this fix the issue for you? Can you explain how you did this?

quidamphx commented 3 months ago

No, there's nothing I can fix by reading about it. I was just describing the behaviour I liked in Windows and how it's not the same in Fedora. Using Xorg, that popup doesn't happen, so that's one way of working around it but I haven't found a way with Wayland.

Whayme commented 3 months ago

Here to report the same behavior on Arch using GNOME.

jarusll commented 2 months ago

Your system information

PC

Steam Beta Branch: Stable Client Steam Version: 1718751621 Steam Client Build Date: Wed, Jun 19 4:09 AM UTC +05.3:30 Steam Web Build Date: Wed, Jun 19 1:36 AM UTC +05.3:30 Steam API Version: SteamClient021 Distribution (e.g. Ubuntu): Fedora Linux 40 (Workstation Edition) Opted into Steam client beta?: No Have you checked for system updates?: Yes GPU: AMD RX 6700 XT

Facing same issue here. I am streaming from Fedora 40 to my Steam Deck(LCD). I am able to stream when I allow to share my desktop with the same modal but steam deck controls do not work. I've tried both controller and KBM.

@aliasbody Did you use PROTON_LOG=1 %command% to generate logs?

Reonu commented 2 months ago

Same behavior here on EndeavourOS with KDE Plasma 6.1 and Wayland. Super annoying.

tlneondo commented 2 months ago

This is also happening under KDE, If I try to use the chord to move the mouse with my controller I get a popup. image If I tell steam input to use a keyboard input on the gamepad I get the same popup again, and worst of all, confirming it doesn't allow the input through, NOR SAVE THAT I WANT THE INPUT TO GO THROUGH.

Which basically breaks steam input's promised controller support for MKB only games.

It is nice that Steam is finally able to move a mouse under wayland, but this is a bit much.

eckce commented 2 months ago

Same issue here with Steam on Gnome with Wayland. Super annoying.

stark-silence commented 1 month ago

Hi just chiming in with this is the most frustrating bug I have encountered in a while. I use a DS4 and even with it completely unbound in Steam's thing it does stuff, so then I thought... wait hold on. Hold on. Hold on one second. So it's doing stuff, even WITHOUT STEAM? HMMMMMMMMM.

So then I remembered, hey, wait I've had this problem myself before. This is Gnome, automatically adding the DS4, as a TOUCHPAD. So at least on GNOME (I am EndeavourOS Gnome atm) you can just press super key, type 'touchpad' bring up the settings menu, click the 'touchpad' tab next to mouse, and then TOGGLE OFF TOUCHPAD.

image

After which you can then go into Steam's settings and set 'trackpads' to just do nothing. This will finally stop your trackpads bringing up some weird esoteric PROBABLY NOT USEFUL menu to you in the middle of trying to S rank Metal Gear Rising or whatever.

This also means I guess you can't use it like a mouse right now if you really wanted to do that, but hey, it'll help for right now if you just want to play video games.

Hope this helps anyone else.

K1ngjulien commented 1 month ago

Having the same "issue" here. Using a controller connected to my TV, everything's fine when I use it just as a controller. When I switch to mouse mode to move the cursor out of the way, I get the "Allow Remote Interaction" popup and it works until I exit the steam link app on my tv.

I say "Issue" because its more of an inconveinience to have to get up to allow mouse movement.

IMO its a valid reason for the popup to show up. Ideally, I'd like to find a way for steam to just always be allowed to do remote interaction. If anyone can help set up some sort of udev(?) config that'd be great.

K1ngjulien commented 1 month ago

As steam uses the remote desktop portal to do its simulated key presses, a fix might be to just allow those sessions to persist?

https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.RemoteDesktop.html#org-freedesktop-portal-remotedesktop-selectdevices

setting the persist_mode to 2 sounds like whats needed here, so steam can reuse the rdp session it requested the last time and i dont have to click on share again.

joaquinvacas commented 3 weeks ago

Since libinput is the default for a few years... isn't libei a better solution for this?

AdrianVovk commented 2 weeks ago

Another instance: https://www.reddit.com/r/gnome/comments/1evs764/remote_desktop_dialogue_shown_when_using_a/

Over there I made a user-friendly writeup of what's causing the bug. I'll copy+paste it over here for reference, with some minimal additional commentary:

Steam takes full control of your gamepad. When you're in a game, Steam receives the events from the gamepad, and forwards them into the Steam game via the Steam Input API. So games with controller support get to see the controller. Games without controller support are remapped into keyboard and mouse events by Steam Input and the game sees these remapped events instead.

When not in a game, Steam Input runs in desktop mode. Steam treats your desktop like a game that doesn't support controllers natively: it remaps the controller into fake keyboard and mouse events, and then tries to send those events into the OS to handle. (There might be more complexity involved here with a component called Gamescope, but that's not relevant to the "controlling your desktop with a gamepad" use-case)

Except, the OS isn't a Steam game. It doesn't have the special Steam Input API to inject fake input events. So Steam has to take control of your desktop some other way.

On X11 there was an API called XTEST that was originally intended for app testing purposes. Like you'd run a script, and that script would run your app and then control the mouse and keyboard to automatically test it. Steam Input (ab)uses this XTEST API to move around the mouse cursor and inject virtual keyboard inputs when it's in desktop mode.

On Wayland no 1:1 alternative to XTEST exists, because it's really insecure to let random apps take control of your desktop. An X11 connection is a trivial sandbox escape because an app can abuse XTEST to take full control of the system, open a terminal app, and then run some commands outside of the sandbox. This is why Flatpak considers X11 a dangerous sandbox bypass, by the way. Wayland was designed to allow security sandboxes, so no such insecure API was added.

But it's a useful feature to have. Things like remote desktop software and Steam do use XTEST for legitimate purposes. So the solution was to make a Portal API that asks the user for permission before letting an app take control of the desktop (and also notifies the user when there's an app controlling their desktop). Apps can now ask for permission to control your mouse and keyboard, and if the user allows it they'll be able to do so.

For backwards compatibility, XWayland provides support for XTEST through the portal. The TXEST API has no concept of asking for permission, so XWayland handles it for the X11 app; whenever the app tries to use an XTEST API to inject a fake event for the first time, XWayland asks the portal for permission - once granted, the app will keep having "remote desktop" control over your system until it's closed. If permission is denied, XWayland just asks again the next time the app tries to use XTEST. There's really no better way to handle this because X11 apps expect XTEST to work and have no concept of needing to ask for permission before taking over your device.

Note that those MRs linked above fix bugs related to handling of this XTEST API with Gamescope running inside of another Wayland session. They don't solve the underlying issue of XTEST clients not being portal aware. They don't have any effect on Steam Input controlling your desktop

The solution is for Steam to talk to the portal natively instead of using XTEST. The portal works on both X11 and Wayland. Steam will then be able to knowingly ask for permission to control the screen, gracefully handle rejection by the user (instead of just spamming with more questions), and so on. It can even opt into letting the OS remember that Steam was previously given permission, and (if the user allows it) the permission will be automatically granted without bothering the user each time.

As noted above, once Steam talks to the portal directly it can opt into a persistence mode to stop asking the user every time. (And it would need to handle storing/retrieving the persistence restore token, of course)

AdrianVovk commented 2 weeks ago

Since libinput is the default for a few years... isn't libei a better solution for this?

Yeah libei is a major part of the remote desktop portal's functionality. So yes, it's part of the solution. But it's not useful to apps without knowledge of the portal (otherwise there's no way to get connected to the ei server).

The real solution is to use the portal in some way. A simple solution is to just use the portal directly to move mouse and emulate keyboard. A more complicated but probably more performant solution is to ask the portal for a libei connection and then send events through that, directly into the compositor.