Installing xpadneo and using Xbox One Controller over Bluetooth flips X+Y buttons and Start+Select buttons stop functioning until uninstalled. Over USB is fine.

ipaqmaster commented 3 years ago

Important note

This is a continuation of my documented experience in this issue: https://github.com/ValveSoftware/steam-for-linux/issues/7531

Continuing template..

Version of xpadneo

Experienced in:

Installing directly from commit 45fef91771bc0d32087c8b66c230417d9249d996
Installing from AUR entry: https://aur.archlinux.org/packages/xpadneo-dkms-git/ which currently provides version 0.9.r83.g45fef91 (Appears to be same as above)

Controller Model

[x] Xbox One S controller
[ ] Xbox Elite 2 controller
[ ] Xbox Series X|S controller
[ ] Other:

Connection mode

[x] Bluetooth connection
[ ] Xbox Dongle connection (not yet supported)
[x] USB cable (not yet supported) [Issue does not happen when I use USB]

Installed Software

[x] Steam Input (enabled by default via Steam Desktop client)
[ ] Steam Link (usually via Raspberry Pi or other micro computers)
[ ] xow (alternative driver using the Xbox dongle)
[ ] Anti-Micro (may affect button mappings)
[ ] netstick (shares input devices via network similar to Steam Link)
[ ] xboxdrv (user-space gamepad driver)

Severity / Impact

[x] I've read the docs and the bug reporting instructions
[x] I've applied the latest firmware update to the controller
[ ] It does not work at all
[x] It used to work in a previous version
[ ] It mostly works but sometimes it doesn't
[ ] I found a work-around
[x] I probably didn't figure it all out but it's too early to give up
[ ] I don't know how to ...
[ ] It's too complicated
[ ] Fantastic work but ...
[ ] I can code and I want to help

Describe the Bug

As title. Installing xpadneo either directly from here or via an available AUR repo package for Archlinux and using my Xbox One Controller over Bluetooth flips the X+Y buttons in all titles. It seems my Start+Select buttons are also ignored. USB is fine but xpadneo claims it isn't supported yet, so using USB is likely just not using xpadneo.

This is only something I wanted to report as this all used to work just fine. Utilities like jstest /dev/input/js0 clearly show the buttons responding so I feel like it's a Steam or xpadneo thing.

I'm not sure if this is an issue of my system configuration, xpadneo, or something like Steam's controller recognition when xpadneo is active. Yet.

I've also noticed Steam will vibrate my controller indefinitely sometimes. I worked around this by disabling rumble for my controller under Big Picture mode in Steam, but ... upon installing xpadneo the infinite rumble bug I'm also experiencing has returned. Further making me wonder if Steam thinks it's a "different" controller.. or something.

Steps to Reproduce

git clone https://github.com/atar-axis/xpadneo
cd xpadneo
sudo ./install.sh
- Boot controller, connect over bluetooth
- Open a game with Xbox controller support
- Experience X+Y flippage and unresponsive Start+Select buttons.

Closing the game, uninstalling xpadneo and reconnecting the controller only to open game of choice again.. and the issue is gone.

Expected Behavior

X and Y buttons not flipped.
Start and Select functional to pause games and access other menus, etc.

(Not confident the infinite rumble bug while Steam running is xpadneo's problem)

Screenshots / GIFs / Videos

System Information

# uname -a
> Linux hostnamegoeshere 5.12.14-arch1-1 #1 SMP PREEMPT Thu, 01 Jul 2021 07:26:06 +0000 x86_64 GNU/Linux

# xxd -c20 -g1 /sys/module/hid_xpadneo/drivers/hid:xpadneo/0005:045E:*/report_descriptor | tee >(cksum)

# $ xxd -c20 -g1 /sys/module/hid_xpadneo/drivers/hid:xpadneo/0005:045E:*/report_descriptor | tee >(cksum)
00000000: 05 01 09 05 a1 01 85 01 09 01 a1 00 09 30 09 31 15 00 27 ff  .............0.1..'.
00000014: ff 00 00 95 02 75 10 81 02 c0 09 01 a1 00 09 33 09 34 15 00  .....u.........3.4..
00000028: 27 ff ff 00 00 95 02 75 10 81 02 c0 05 01 09 32 15 00 26 ff  '......u.......2..&.
0000003c: 03 95 01 75 0a 81 02 15 00 25 00 75 06 95 01 81 03 05 01 09  ...u.....%.u........
00000050: 35 15 00 26 ff 03 95 01 75 0a 81 02 15 00 25 00 75 06 95 01  5..&....u.....%.u...
00000064: 81 03 05 01 09 39 15 01 25 08 35 00 46 3b 01 66 14 00 75 04  .....9..%.5.F;.f..u.
00000078: 95 01 81 42 75 04 95 01 15 00 25 00 35 00 45 00 65 00 81 03  ...Bu.....%.5.E.e...
0000008c: 05 09 19 01 29 0c 15 00 25 01 75 01 95 0c 81 02 15 00 25 00  ....)...%.u.......%.
000000a0: 75 01 95 04 81 03 05 0c 0a 24 02 15 00 25 01 95 01 75 01 81  u........$...%...u..
000000b4: 02 15 00 25 00 75 07 95 01 81 03 05 0c 09 01 85 02 a1 01 05  ...%.u..............
000000c8: 0c 0a 23 02 15 00 25 01 95 01 75 01 81 02 15 00 25 00 75 07  ..#...%...u.....%.u.
000000dc: 95 01 81 03 c0 05 0f 09 21 85 03 a1 02 09 97 15 00 25 01 75  ........!........%.u
000000f0: 04 95 01 91 02 15 00 25 00 75 04 95 01 91 03 09 70 15 00 25  .......%.u......p..%
00000104: 64 75 08 95 04 91 02 09 50 66 01 10 55 0e 15 00 26 ff 00 75  du......Pf..U...&..u
00000118: 08 95 01 91 02 09 a7 15 00 26 ff 00 75 08 95 01 91 02 65 00  .........&..u.....e.
0000012c: 55 00 09 7c 15 00 26 ff 00 75 08 95 01 91 02 c0 05 06 09 20  U..|..&..u......... 
00000140: 85 04 15 00 26 ff 00 75 08 95 01 81 02 c0                    ....&..u......
3560615638 1558

Controller and Bluetooth Information

xpadneo-dmesg.txt xpadneo-btmon.txt xpadneo-lsusb.txt

Additional Context

Toggling Steam Beta Participation and restarting Steam changes nothing.

Unticking Xbox Controller Support in Big Picture just renders my controller unusable in games.

kakra commented 3 years ago

That your SDL string starting with 0300 indicates that your controller is used via hidraw in SDL. This is currently known broken.

To confirm this problem, try the following steps:

Connect the controller and watch dmesg
Get the name/number of the hidraw device from the log, it will look like: xpadneo 0005:045E:0B13.0019: input,hidraw15: BLUETOOTH HID v9.03 Gamepad ... (hidraw15 in my case)
Check the permissions of the file /dev/hidraw15 (from my example) and post them
Revoke the permissions: sudo chmod a-rwx /dev/hidraw15 and try your games now, ensure the controller stays connected otherwise the device file will be recreated, usually with a new name

If this is fixes your problem, our work-around doesn't seem to apply. It used to work in previous versions then because SDL2 wasn't using the hidraw device back then.

Cross references:

kakra commented 3 years ago

Unticking Xbox Controller Support in Big Picture just renders my controller unusable in games.

It works just fine for me. But well, the main problem may be the hidraw support in the first place. How does it render the controller unusable?

ipaqmaster commented 3 years ago

Thanks for getting back to me on this.

For clarification, when unticking Xbox Configuration Support in Big Picture it seems to be hit and miss for which titles will work. For example, Mortal Kombat 11 running through GloriousEggroll's Proton 6.1 works with it unticked in Big Picture but Linux-native game "One Step From Eden" can see the controller present but does not respond to controller input anymore.

As for your instructions above for the hidraw device:

[  +0.000095] microsoft 0005:045E:02FD.0026: input,hidraw11: BLUETOOTH HID v9.03 Gamepad [Xbox Wireless Controller] on 84:c5:a6:5a:5f:81
[  +0.000061] xpadneo 0005:045E:02FD.0026: input,hidraw11: BLUETOOTH HID v9.03 Gamepad [Xbox Wireless Controller] on 84:c5:a6:5a:5f:81

me@desktop ~ $ ls -lah /dev/hidraw11 
crw-rw----+ 1 root root 241, 11 Jul 17 20:44 /dev/hidraw11
me@desktop ~ $ sudo chmod a-rwx /dev/hidraw11
me@desktop ~ $ ls -lah /dev/hidraw11 
c---------+ 1 root root 241, 11 Jul 17 20:44 /dev/hidraw11

Trying OSFE again after revoking hidraw11's permissions with chmod the game accepted no controller input. I unticked Xbox Configuration Support in Big Picture again and still no change on relaunch.

MK11 was happy to use the controller after revoking the permissions of /dev/hidraw11 and also even after unticking Xbox Configuration Support in Big Picture and trying again.

It seems to be hit and miss? Though I wonder if Proton helps MK11 being a Windows-only title.

kakra commented 3 years ago

There are a few titles which actually need Steam Input to properly work with all controllers. Usually, such titles come with an override option set by default in the game properties. I think Horizon Zero Dawn is such a title.

c-x-berger commented 3 years ago

I'm seeing something similar. Immediately after installing xpadneo, Steam's "Big Picture" mode has some interesting swaps for me:

Pressing Y registers as pressing X
Pressing LB registers as pressing Y
Start and Select map to the nearest bumper (I genuinely don't know which is Start and which is Select, sorry)
I have no idea where start and select have been moved to, but it's probably just as cursed.
A and B behave correctly.

If "Xbox Controller Support" is enabled in Big Picture mode, two things happen:

The mismatched buttons carry over into whatever game I start. Probably because something something Steam Input something.
Shortly after starting the game I've used for testing (Hyper Light Drifter), one of the rumble motors will begin going at max power and doesn't stop until Steam itself is closed. So far it's usually been the force feedback on the right trigger, but it has happened on the left trigger as well. (Haven't yet seen it on the "main" rumble motors.)

None of this happens when connected over USB.

If I remove the module with sudo dkms uninstall -m hid-xpadneo -v v0.9-85-g67585b3 (because uninstall.sh seems to be broken), buttons seem to map properly.

kakra commented 3 years ago

@c-x-berger This is probably because SDL uses the hidraw device directly. Our HID report format is a bit different than what SDL expects (because xpadneo uses the same bitmap format across all controller models and SDL doesn't look at the report size to decide for a bitmap format but at the VID/PID instead, and then it even looks at the wrong one which we cannot work around). This also explains why the rumble goes crazy: xpadneo has a protection against a race condition in the controller firmware but we cannot apply that in the hidraw layer. If rumble data is sent too fast, the motors will stop responding to commands and spin for minutes before they eventually stop. The controller firmware may even reboot or not recover from that at all until you pull the batteries.

I quick fix would be to look at dmesg after connecting the controller to find the number of the hidraw device, then chmod a-rwx /dev/hidrawNUMBER, then see if the problem persists. If the controller is disconnected, the chmod change is lost and needs to be re-applied (the hidraw number usually changes at that point).

I'm currently planning on re-implementing the whole hidraw interface to be compatible with SDLs assumptions but we would probably still not be able to protect against the rumble race. But as far as I know that's fixed in newer SDL versions.

Buttons X and Y shift by one position because different Xbox controller models have different sparse bitmap formats, even within the same model depending on some strange firmware quirks. The original wired controller has bits A,B,X,Y,LB,... for the buttons, the wireless controllers have A,B,C,X,Y,Z,LB,... where C and Z are even not buttons on the gamepad. That's why the buttons move. The buttons that come after the initial bits may be even more messed up.

This has been documented here: https://github.com/atar-axis/xpadneo/issues/286

It's for historic reasons why xpadneo handles the device the way it does. With current SDL versions, this is probably no longer necessary. But I'm planning on adding wired and dongle support in HID mode and need to find a way to cover all cases.

Please report a uninstall bug separately - including the output you get.

kakra commented 3 years ago

If "Xbox Controller Support" is enabled in Big Picture mode, two things happen:

One more idea: Update the controller to the latest firmware, SDL may apply wrong assumptions about the model if the firmware is too old (because MS messes with the HID report format between firmware versions, and SDL absolutely doesn't care about HID descriptors at all, it just assumes nothing ever changes in the format).

kakra commented 3 years ago

I have no idea where start and select have been moved to, but it's probably just as cursed.

Probably to the thumb stick buttons and the guide button.

BTW: Select is left, start is right. As a memory hook: left goes back (to the selection menu), right goes forward (starting the game).

tb936 commented 3 years ago

I am also experiencing this issue, have downloaded the latest firmware from the xbox app. The sticks don't map to start and select as I would expect.

Will use the hidraw device workaround until this is cleaned up

kakra commented 2 years ago

This should actually be fixed, please confirm.

Andrew-Fahmy commented 2 years ago

I was having a similar issue with the buttons being mapped wrong in steam games. This was due to the hidraw device having the incorrect permissions which allowed SDL to read the device. This would fix my problem until the controller is disconnected:

I quick fix would be to look at dmesg after connecting the controller to find the number of the hidraw device, then chmod a-rwx /dev/hidrawNUMBER

I did some more digging and found out that there was a udev rule that was overriding the one that was installed by xpadneo. I had installed a rule into /etc/udev/rules.d/ which overrides any of the rules in /lib/udev/rules.d/. After removing that rule (or moving it to /lib/udev/rules.d/) my controller now works perfectly.

If anyone else if having issues, here is how I was able to find what rules where being loaded:

Disconnect your controller

Set udev log to debug and watch journal log

udevadm control --log-priority=debug
journalctl -f

Reconnect controller

This should give you a little more info about what rules could be interfering. Also If you do make changes to any rules make sure to reload the changes with udevadm control -R. Hope this helps :)

kakra commented 2 years ago

If anyone else if having issues, here is how I was able to find what rules where being loaded:

@Andrew-Fahmy Can you remember which one actually has been the conflicting rule?

I think the following may happen:

Someone installs xpadneo from source, which installs rules to /etc prefix. Then, this installation is replaced by a distribution package without uninstalling the manual installation first: This will keep old /etc files but also install identically named files in /usr/lib prefix. Now, udev has a rule which allow rule file names to exist only once, and it prefers admin configuration in /etc over distribution configuration in /usr/lib (which makes sense). This ultimately puts you into the position to properly clean up a mess you installed as a local admin. ;-)

Maybe we'd need some xpadneo doctor which can check for this situation.

Andrew-Fahmy commented 2 years ago

In my case I manually installed the conflicting rule from qmk firmware as their qmk doctor command recommended. It has a rule that would set the mode of the all hidraw devices so that a tool called hid_listen could print out debug info coming from the keyboard with qmk firmware.

Do you think that I should open an issue about this with qmk?

Also I think an xpadneo doctor would be a nice addition.

kakra commented 2 years ago

Actually, this line should not be there:

KERNEL=="hidraw*", MODE="0660", GROUP="plugdev", TAG+="uaccess", TAG+="udev-acl"

It's a security mess, don't make hidraw device just readable to every user in a group (plugdev) or every user logged into a local session (uaccess), and I'm not even sure what "udev-acl" is supposed to do. This rule is bad. It allows any local process in your session to spy on any hidraw device (even when unrelated to your session), which is almost every input device in the world, thus allowing to record keypresses, pin entries, or more generally: secrets. Don't do that.

Meanwhile, in Discord, we discovered that OpenRGB installs a similar problematic rule.

Access to hidraw devices MUST be guarded by conditionals for the specific devices which should get this ACL (uaccess).

Andrew-Fahmy commented 2 years ago

That is a really good point. I had not thought about it from the security perspective. I'll open an issue with qmk about this.

For now what do you think about adding some info about the udev rules to the troubleshooting page?

Also I want to thank you for all the work you have done on this project and responding to all my questions and concerns quickly and thoroughly!

kakra commented 2 years ago

Actually, we can make it compatible even with exposed hidraw device. This conflict just uncovered questionable security issues in other software.

The change to re-enable xpadneo hidraw and make in compatible with SDL is planned for v0.10 or v0.11. Until then, xpadneo doesn't add uaccess rules to hidraw. But if other software does it unconditionally, it conflicts with our configuration - no matter whether it's a security problem. So it's bad already from two different views.

I'm pretty sure it's an oversight. It should be fixed or at least considered in some way, no matter if xpadneo managed to work around it or restored SDL-hidraw compatibility. OTOH, I don't like the hidraw approach of SDL very much, it re-implements logic that's already done at the kernel level, without proper checks if mapping fixups were already done (i.e., it doesn't look at the HID descriptor, it just assumes some format of the HID reports).

kakra commented 2 years ago

For now what do you think about adding some info about the udev rules to the troubleshooting page?

I'll follow up on this with a PR when we fully understand the problem. I'm pretty sure we almost see the whole picture. I now need some confirmation that uninstalling those rules (aka the package that installed them) actually fixes what we are seeing. It's a long standing bug, I don't know yet if it is the same problem in each issue because we have no documentation about what other software is installed, or which custom udev rules may exist. We are probably running races we cannot win, I'd need to fix our hidraw reports anyways (which makes controller mapping fixups much more complicated in xpadneo).

Andrew-Fahmy commented 2 years ago

It should be fixed or at least considered in some way, no matter if xpadneo managed to work around it or restored SDL-hidraw compatibility.

They have fixed it, they have another repo dedicated to generating the rules that I was not aware of

I now need some confirmation that uninstalling those rules (aka the package that installed them) actually fixes what we are seeing.

For me removing those rules fixed all the problems

kakra commented 2 years ago

This is exactly how OpenRGB should also fix their world-accessible hidraw rule.

naynayll2 commented 2 years ago

For now what do you think about adding some info about the udev rules to the troubleshooting page?

I'll follow up on this with a PR when we fully understand the problem. I'm pretty sure we almost see the whole picture. I now need some confirmation that uninstalling those rules (aka the package that installed them) actually fixes what we are seeing. It's a long standing bug, I don't know yet if it is the same problem in each issue because we have no documentation about what other software is installed, or which custom udev rules may exist. We are probably running races we cannot win, I'd need to fix our hidraw reports anyways (which makes controller mapping fixups much more complicated in xpadneo).

I can confirm that uninstalling qmk rules fixed the problem for me as well.

kakra commented 2 years ago

Deferring this to v0.11 as it is probably about hidraw interaction with OpenRGB and SDL. This can probably be solved by #286.

Chr1sNo commented 2 years ago

This is exactly how OpenRGB should also fix their world-accessible hidraw rule.

Some feedback from upstream. The blanket hidraw rule was removed when udev rules were moved to compile time generation. This assumes that there are no xpadneo devices that would require an OpenRGB rule.

I would test to confirm this however I don't have a relevant XBone controller so please let me know if this is still an issue with OpenRGB.

carsoncall commented 2 years ago

In my case I manually installed the conflicting rule from qmk firmware as their qmk doctor command recommended. It has a rule that would set the mode of the all hidraw devices so that a tool called hid_listen could print out debug info coming from the keyboard with qmk firmware.

Came here to bump this, I have installed QMK relatively recently and the rule that Andrew-Fahmy mentioned was present in my /etc/udev/rules.d/50-qmk.rules though I do not recall manually installing it. I am led to believe that it is now a default in QMK, at least version 2022-02-26. Simply commenting it out fixed my issues.

It is worth mentioning that I am relatively inexperienced with these things and the documentation makes no mention of this, so I spent quite a bit of time messing with configuration and environmental variables before finally finding this bug report. If it is the right thing, I would be willing to open a separate bug report for docs or contribute the fix myself.

kakra commented 2 years ago

Came here to bump this, I have installed QMK relatively recently and the rule that Andrew-Fahmy mentioned was present in my /etc/udev/rules.d/50-qmk.rules though I do not recall manually installing it. I am led to believe that it is now a default in QMK, at least version 2022-02-26. Simply commenting it out fixed my issues.

According to the commit discussions around that topic, this file should no longer be needed and there should be an identical named file in /usr/lib/udev/rules.d which does things properly. Maybe the needed commits didn't make it into a stable version already?

User settings for QMK should go to /etc/udev/rules.d/49-qmk.rules instead.

As far as I understand, the conflicting part of the rule is about some debugging console only, so if you're not a developer, you should not need it anyways, and commenting it out should not impact your usage of the software.

I suggest moving this part of the discussion to your distribution tracker or QMK itself, as I don't use that software and cannot support it. Alternatively, you may find users of QMK in our Discord channels. If you continue discussion somewhere else, feel free to leave a link here, so other users finding their way here can follow the link. I also accept PRs which add instructions to the docs. Thanks. :-)

atar-axis / xpadneo