search

libretro / RetroArch

Cross-platform, sophisticated frontend for the libretro API. Licensed GPLv3.
http://www.libretro.com
GNU General Public License v3.0
10.38k stars 1.84k forks source link

Linux/Wayland-Native - Multi mouse support #16886

Open SeongGino opened 3 months ago

SeongGino commented 3 months ago

Description

On Linux in a native Wayland session (simply running retroarch) with any input plugin (so udev or wayland), RetroArch does not seem able to support multiple mice - furthermore, trying to use any pointer device (whether it be absolute positioning lightgun or normal relative mouse) will not move the point of aim/aiming reticle in lightgun games.

To work around this, running RetroArch in XWayland is needed - in my case, using WAYLAND_DISPLAY="" retroarch forces RA to run in XWayland, and I can have access to a fully functioning udev input driver with multiple mice support.

Mouse Devices list under native Wayland (either udev or wayland input driver): 2024_08-16 124704

Mouse Devices list under XWayland with udev input driver: 2024_08-16 124815

In both examples, a Logitech G502 mouse and an OpenFIRE Lightgun are plugged into the PC, running under a user in the input group.

Expected behavior

At minimum, the ability to use a mouse device in lightgun games on a native Wayland RetroArch session; ideally, udev or wayland input driver should have feature parity with udev driver under X11 with the ability to address and use individual pointer devices for multiplayer.

Actual behavior

No mouse input is allowed in cores in either udev or wayland input drivers on native Wayland RetroArch.

Steps to reproduce the bug

  1. Input driver should be set to udev
  2. Run RetroArch in native a Wayland session (retroarch)
  3. Go to Settings->Input->RetroPad Binds->Port 1 Controls->Mouse Index
  4. Observe there only being a "Default" index, with all 15 other slots being "N/A"
  5. Launch a game in a core that supports Lightguns, e.g. MAME (Current)
  6. Observe how mouse button inputs are being recognized, and the cursor remains exactly in wherever the default position is regardless of mouse input.
  7. Exit, and relaunch RetroArch under XWayland (WAYLAND_DISPLAY="" retroarch)
  8. Repeat step 2
  9. Observe the Mouse Index being populated by multiple devices (if plugged in) with real names.
  10. Repeat step 4
  11. Observe how the in game aiming reticle is affected by input, depending on the mouse index setting.

Bisect Results

N/A (has this always been an issue?)

Version/Commit

Environment information

SeongGino commented 3 months ago

Small correction: XWayland doesn't solve the issue very well either, as the core cursor is disconnected from the system's in a way that isn't normal for regular X11 functionality.

Using Windowed mode just for sake of demonstration with a visible cursor, this still applies in fullscreen--note how MAME's blue aim reticle is nowhere near the system's black cursor. 2024_08-16 215315c In practical terms, I can't aim beyond the bottom-left quadrant of the game screen area without it snapping to offscreen.

gouchi commented 3 months ago

Similar issue for X11 input driver.

zoltanvb commented 2 months ago

I was under the impression that Wayland only supports 1 system-wide cursor, but failed to find a normative reference, instead I ran into a couple of old issues telling that for graphics tablets, there is actually a second cursor.

When graphics context driver is Wayland, input driver will also be forced to Wayland, it cannot be changed to udev (even though it may appear in the menu like that).

SeongGino commented 2 months ago

I was under the impression that Wayland only supports 1 system-wide cursor, but failed to find a normative reference, instead I ran into a couple of old issues telling that for graphics tablets, there is actually a second cursor.

When graphics context driver is Wayland, input driver will also be forced to Wayland, it cannot be changed to udev (even though it may appear in the menu like that).

I'm not particularly well-versed in the differences of the display servers, but where would you say the root cause of the problem is?

Basically, is this at the fault of Wayland itself; and if so, perhaps where would this kind of issue be posted for more constructive resolution? Wayland itself? The Plasma compositor?

And ideally, I'd at least like the XWayland fallback to still work, but not sure if that's also another Wayland-specific issue rather than the fault of RetroArch. For as good as Wayland mostly is elsewhere, lightguns are the single biggest blocker for me at the moment (which makes things no less easier when you're also developing a lightgun system) and RA has been my goto for virtually everything that isn't a Wii, PS2/3, or Windows-based Arcade title.

Now that I think about it, I've noticed in Wine that the behavior with XWayland is incredibly similar - relative mouse input is okay, but any abs/gun coords are ignored. Honestly, the more I've been thinking about this, the more my head's being done in.

(Just to be clear, any frustrations being vented here aren't with RA itself, but more just exasperated at the general state of Wayland for my inconveniently hyper niche usecase)

zoltanvb commented 2 months ago

I found something that may explain the single mouse device better (not sure if it is up to date): https://wayland-book.com/seat.html The lightgun related part of the wayland input driver is scattered with todo's, so there may be some missing parts also on the RetroArch side of the court. https://github.com/libretro/RetroArch/blob/98c79b3f146938a44205418b83793e85af5202b8/input/drivers/wayland_input.c#L365

zoltanvb commented 2 months ago

I have checked it in a bit more detail, I believe it could be improved a bit on the Wayland input driver side, I'll just need to set up some kind of test tooling so that I can reliably verify different scenarios.

gouchi commented 2 weeks ago

@SeongGino Can you try to make a test with development build or perhaps AppImage if it is working in your environment Wayland + KDE ?

Thank you.

SeongGino commented 2 weeks ago

As far as I can tell, positioning seems to work with current devel ae20251--though, the aux buttons don't seem to be mapped appropriately in-game? Gun Aux A seems to be detected (according to MAME Current) as Gun Start on the Wayland input driver.

zoltanvb commented 2 weeks ago

Button mapping is still a bit messy. Some buttons are hard mapped to mouse buttons left/right/middle/left+right, but the rest (including AUX) is just not working yet, even if it has e.g. a keyboard button assigned.

SeongGino commented 2 weeks ago

Understandable! Just wanted to make sure if the mapping is--well, not intentional but that it's a known issue. I also noticed that the Wayland driver is hardcoded to only use the first lightgun device atm, so I figured that probably didn't warrant mentioning until it's supposed to be working. c:

zoltanvb commented 2 weeks ago

How would be there more lightgun devices under Wayland? (Don't get me wrong, it is not a rhetorical question, I may be overlooking something.)

As far as Wayland is concerned, there is no "lightgun", only pointer, and touch device, for the actual "seat". RetroArch as a frontend can map the single pointer coordinates to "lightgun" device, when a core asks for this specific input type. The core can ask for the input of player (port) 2, 3 etc. lightgun, but in this case, the frontend has 2 options: only return values for the 1st (this is what is done currently), or return same values for all, which would not be more useful I guess. Unless the 2nd player (port) is interpreted as a different Wayland seat, but my impression was that it is a rarely used feature.

SeongGino commented 2 weeks ago

How would be there more lightgun devices under Wayland? (Don't get me wrong, it is not a rhetorical question, I may be overlooking something.)

What I'm talking about and asking for is parity with the udev or Windows rawinput drivers that can access two-or-more separate mouse devices separately - as the wayland driver is now in the last addressed commit, it still shows as Pic 1 in the OP (only displays one "Default" mouse device, nothing else). Speaking as a lightgun firmware dev, this is a FAQ we're constantly asked about by users, so I know it's going to be something people will want. If people won't have this functionality in Wayland, then they'll have to use a non-Wayland display solution instead.

FWIW, most of the "how Wayland works" stuff goes over my head; as interesting as the nuts-and-bolts aspects are, I'm really only interested in what affects users. Apologies for the misunderstanding.

zoltanvb commented 2 weeks ago

OK, the request is clear. I just dump some info here for any future reference:

SeongGino commented 2 weeks ago

I see. So it looks like, at least as of now, reliable multiple pointers (particularly Absolute positioning) requires manual setup that isn't needed on X11--and even then, it's not consistent if it even works across every distro.

Sounds like it might be a bit of a pain, which seems pretty on-brand for Wayland from what I'm noticing. :/