Closed ehats closed 4 years ago
ehats
commented
4 years ago I also just spotted your new PR updating the docs and I tried one of the suggestions there:
jscal -u 8,0,1,3,4,2,5,16,17,10,304,305,307,308,310,311,314,315,317,318 /dev/input/js0
jscal: joystick has 10 axes and not 8 as specified on command lineIt worked perfectly when connecting over USB though.
kakra
commented
4 years ago @ehats Yes, this fixup is currently only for the XB1S controller (or other variants with the same layout).
I wonder if the XBE2 controller really has 8 physical axes... Are those two axes additional analog triggers at the bottom of the device? Or are those also just digital buttons?
Also, I wonder what all those buttons are... Do they have a function? Which physical button is mapped to it?
Pressing A activates buttons 0 and 10. B activates 1 and 12. X activates 3 and 14. Y activates 4 and 15.
We need to remove all the duplicate buttons from the report descriptor. Please make a complete list of all duplicates. Continue to use the numbering from the screenshot.
SELECT doesn't work.
Yes, this is probably because between 9 and 11 there's an additional button 319. For the jstest to work correctly, we are required to expose exactly 10 buttons, not 11. I'm not sure what happens if we have even more buttons.
MENU activates 22.
Using the D-pad affects axes 7 and 8 as expected, but it also influences axis 6 in a seemingly random way.
This is a mapping problem that can be fixed by re-ordering the mapping with jscal -u. Apparently, jstest-gtk doesn't look at the axes names when mapping the axes to UI elements, it just goes by the ordering number. But this is true for most joydev-only games, too. In that case, you may need to fix the ordering but if you're going to use evdev games, just ignore jstest-gtk mapping.
We need to work on this in multiple phases and I hope you'd like to support me with this. First, we need to fix the ordering of the axes and buttons to comply with evdev standards, ignoring erratic test results of the axes in jstest-gtk, let's use jstest on the command line instead during this phase. We probably need to reorder 6: ABS_RUDDER and 9: ABS_MISC after the hat switches. What do 6 and 9 physically correspond to?
In phase two we are going to ensure that the fixups work correctly with SDL2 and Chrome gamepad API, and that games actually see the correct button ordering. We'll check if it works with exactly 10 buttons only (and the others mapped to keyboard events) or if it also works with 12 buttons or more. 11 buttons doesn't work correctly, games don't expect to see the Xbox button as a joystick button, it has a special meaning and should not be available to games.
In phase three we'd adjust the docs to support a jscal -u command for this controller. I'll probably let you run some tests with udevadm.
ehats
commented
4 years ago Thanks for the response! I fiddled around with the buttons and axes a bit to figure out what's what. Here's what I found:
Axes 0-5 seem to be in order. They're LX, LY, LT, RX, RY, RT
Axis 6 looked very random at first, but I think I figured it out: it's the "angle" of the D-pad. Pressing up sets it to -32676, pressing left sets it to 32767. The value actually updates at 45 degree intervals, so down+right gets you zero. Up+left doesn't seem to do anything (it'll be +/- 32676 depending on which one was closer). Also: releasing the D-pad doesn't clear the value, it remembers the last position.
Axes 7 and 8 are D-pad X and Y.
Axis 9 seems to move with axis 0, albeit with a few frames lag.
Example from jstest while moving the left analogue stick (center-right-left-center):
jstest-l1.log
You also asked about the bottom of the gamepad. There are four buttons, which don't do anything in the default profile. Setting the gamepad to profiles 1/2/3 enables them as button 0,1,3,4 (same as ABXY which also skip button 2 for some reason).
Here's a mapping based on jstest (what I pressed -> which buttons were activated) A: 0, 10 B: 1, 12 X: 3, 14 Y: 4, 15 Select: nothing Home: 22 Xbox button (guide): nothing LB: 6, 17 RB: 7, 18 Left stick: 24 Right sick: 25
Paddles on the bottom (again, these only work if not using the default profile): P1: 1 P2: 4 P3: 0 P4: 3
As far as I know, it has never been connected to a windows device or configured in any way (and I bought it as new). And again, connecting via USB only reveals 8 axes and 11 buttons, but I'm not quite sure what that really means.
kakra
commented
4 years ago Axes 0-5 seem to be in order. They're LX, LY, LT, RX, RY, RT
Yes, this looks correct. But we can only continue if you didn't update the joydev mapping yet (either by using jscal -u or from within jstest-gtk. If you did, clear /var/lib/joystick/joystick.state and reconnect the controller. In this case, you'd need to redo the tests. I'm assuming we are running with vanilla joydev mappings (aka "not customized").
Axis 6 looked very random at first, but I think I figured it out: it's the "angle" of the D-pad. Pressing up sets it to -32676, pressing left sets it to 32767. The value actually updates at 45 degree intervals, so down+right gets you zero. Up+left doesn't seem to do anything (it'll be +/- 32676 depending on which one was closer). Also: releasing the D-pad doesn't clear the value, it remembers the last position.
Okay, that axis shouldn't be there then. It will confuse games.
Axes 7 and 8 are D-pad X and Y.
LGTM but should be 6 and 7.
Axis 9 seems to move with axis 0, albeit with a few frames lag.
No idea where that comes from and why it does that, I'll have to investigate the report descriptor you posted and remove it from there.
Example from
jstestwhile moving the left analogue stick (center-right-left-center): jstest-l1.log
Funny, even jstest doesn't know what to make out of all these buttons, or what this mysterious laggy last axis is.
The log looks strange: From your description it should only see changing numbers on axis 0 (let's just ignore axis 9 for the moment). But it seems that also the trigger axes 2 and 5 started to change. I can only guess that this is a calibration issue: joydev auto-calibrates during usage: If you ever so slightly touched the triggers during test, it will start showing numbers. Maybe collect some calibration data first in the next tests by moving all axes and triggers to their maximum and minimum positions, and only then create logs with jstest. This needs to be redone after every reconnect (there're ways to make calibration data and mappings persistent over reboots but we don't want to do that right now).
You also asked about the bottom of the gamepad. There are four buttons, which don't do anything in the default profile. Setting the gamepad to profiles 1/2/3 enables them as button 0,1,3,4 (same as ABXY which also skip button 2 for some reason).
So these are the "doubled buttons" you described earlier? I think we should deal with those later and first try to get all the standard buttons working (that is, only 10 buttons, the Xbox button excluded). Also, we have to find out about the profiles: Do you know if the default profile is supposed to make the paddles work at all? If yes, we should work with the standard profile from now on. A btmon log could show you if the paddles expose activity in the default profile. If they do, let's work with the default profile from now on and figure out how to map those properly.
Here's a mapping based on jstest (what I pressed -> which buttons were activated) A: 0, 10 B: 1, 12 X: 3, 14 Y: 4, 15
I'm not sure what to do about 10, 12, 14, and 15. Maybe those are the paddles? Do those buttons show activity in the default profile? And using the default profile, do you see changes in btmon when using the paddles?
Select: nothing Home: 22
This problem sounds familiar to me after I updated my XB1S firmware lately. It's working correctly again in master.
Xbox button (guide): nothing
Yes, this should be a keyboard event. You should be able to see it when using evtest. But that may depend on the previous two buttons.
LB: 6, 17 RB: 7, 18
These may also be duplicated by using the non-standard profile. Or the controller exposes two bit-arrays, one for backwards compatibility with XB1S, and another bigger array for the new buttons. We can investigate that by looking at the btmon results, pressing one button at a time, looking at the log and annotating it with the button pressed and released.
Left stick: 24 Right sick: 25
I guess you mean the thumb stick buttons (when depressing the sticks)? These should not be at those positions, these should be 8 and 9.
Paddles on the bottom (again, these only work if not using the default profile): P1: 1 P2: 4 P3: 0 P4: 3
I'm just assuming the driver cannot currently map those in the default profile. But if you see them in btmon in the default profile, we will be able to get these working.
As far as I know, it has never been connected to a windows device or configured in any way (and I bought it as new). And again, connecting via USB only reveals 8 axes and 11 buttons, but I'm not quite sure what that really means.
Okay, let's stick with that situation and do not yet connect it to Windows, so that we can create a proper OOTB experience for Linux-only users. Also, connecting via USB is meaningless, as that doesn't use the xpadneo driver. But connecting via USB may switch the Bluetooth compat mode of the firmware. You probably should un-pair and re-pair the controller twice after having it connected via USB to return it to its Linux Bluetooth mode properly.
Currently, it looks more useful if you start testing with evtest first, because HID is the native protocol used by the controller in Bluetooth mode. The joydev interface only sits on top of that in an emulation layer of the kernel. So we need to fix evdev first, then work our way up to jstest.
Let's start with btmon as it reports the raw data coming from the device before any processing by the input layer. Try the default profile, press and release a button, then copy each packet to a text file annotating each event with the action you took (pressed A, released A, ...). Don't post the complete btmon log as it has an overwhelming amount of data, I just need the single events for buttons. Also, write down if a button did nothing in btmon when using the default profile. I know this is a lot of work but it will help the community and me a lot. So thanks in advance for your efforts.
ehats
commented
4 years ago /var/lib/joystick/joystick.state did not exist, so I just unpaired and reconnected (which I have to do an awful lot anyway)
I guess you mean the thumb stick buttons (when depressing the sticks)?
Yes, never really sure what to call those at times...
Anyhow, here you go, I captured the packets for all of the buttons (except the thumb sticks as it's a lot more hassle because of the sensitivity (see below), but happy to do it after processing these :)).
A couple of discoveries:
Even though I didn't include messages for axes, triggers and the d-pad, I did see messages for everything (including the Guide button)
Hope this is enough to get started and let me know what we'd need next!
kakra
commented
4 years ago This is fantastic, kudos for the discovery of the profile data. It will take me some time to tell all the fields apart from each other and put it into a table I can more easily work with.
From what I see, we can easily discover the paddle buttons from the byte stream.
I'm yet unsure how to handle the profile number, maybe we opt-in for supporting profile 0 only as a first start and discard reports from other profiles. Otherwise, we add complexity to the driver too early.
And yes, the sticks are really sensitive already on the XB1S. It always helped me to rest the controller on the table and firmly holding it in place while pressing buttons. This doesn't make it easier for the thumb buttons, tho. But at this point I'll just stick (haha, you get it?) to guessing which bits show up for those.
ehats
commented
4 years ago Okay, this was a bit more fun than I expected and I ended up documenting a few more things, maybe this is helpful.
a1 01 LX LX LY LY RX RX RY 87 LT LT RT RT DP BB
ST SE LX LX LY LY RX RX RY 87 LT LT RT RT DP BB // Copy of the first row (except first two bytes)
ST SE PP 00 PR TS 00 00 00 00 00 00 00 00 00 00 // First two bytes are a copy of the first two bytes from above
00 00 00 00 00 00 00 00
LX, LY, RX, RY: X and Y axes for the left and right sticks (00 00 - ff ff)
LT, RT: Going from 00 00 to FF 03
DP: D-Pad info
0 -> nothing
01-08 -> N, NE, E, SE, S, SW, W, NW
BB: on/off for ABXY, LB, RB
0bRL0Y X0BA // R = RB, L = LB
If Profile != 00, then ONLY THE FIRST ROW's (!!!) BB are on/off values for the paddles:
0b0002 4013 // 1 = P1, etc.
ST: on/off for Start and Sticks (thumbstick buttons)
0b0000 SRL0 // S = Start, R/L = Right/Left thumbstick button
SE: on/off for Select
0b0000 000S // S = Select
PP: on/off Paddles
0b0000 4321 // 1 = P1, etc.
PR: Profile
00-03 -> Number of LEDs on (= Profile number)
TS: Trigger settings
0b0000 RrLl // R/L = RT/LT has 2 states, r/l = RT/LT only goes halfway.
Pressing the connection button sends a new packet, but there's nothing in the packet about itI already decoded the format some time ago without access to packet dumps like you now have, by looking at the HID descriptor. Maybe you want to work on that file: docs/descriptors/xbe2_linux.md. Actually, the bit positions, sizes, and meanings all come from the descriptor. But be warned: For the table you either need a wide monitor or a small font. ;-)
The two bytes in the beginning have the following meaning:
a1 is a protocol byte in the Bluetooth layer. We can ignore that as we don't see it in xpadneo. a1 is received HID data, a2 is sent HID data (i.e., rumble data).
01 is the HID report identifier. This is the first byte xpadneo is seeing. It matches with the first column in the table of that file I mentioned initially. Battery reports have 04, rumble packets have 03 in their ID.
It seems the XBE2 controller even has a programming interface, probably for LED programming and other purposes (probably report IDs 06 and 09).
I've built the table from the decoded HID descriptor in the beginning (there's an online decoder, you should first check if your descriptor matches, otherwise create a new file).
The button map at the end of the file is incomplete (it's actually just a copy from XB1S currently).
It seems the descriptor is quite messy and just abuses the media control stuff for buttons and profiles. This will need a lot of fixup code.
ehats
commented
4 years ago I've run xxd -c20 -g1 /sys/module/hid_xpadneo/drivers/hid:xpadneo/0005:045E:0B05.0010/report_descriptor, then stripped the address and ASCII bits and ran it through the tool. The results were indeed different. I'd be happy to create a PR for it, but I'm not quite certain what to call it, since it's also an XBE2 (except I'm not sure if it is in Linux mode, but I guess that's somewhere in the logs I uploaded).
kakra
commented
4 years ago If unknown, call it unknown. ;-)
And no, we do not know yet if there's even different modes (for XB1S, each mode has a different product ID except for one edge-case). It could also be that it's just a different firmware version.
Also noteworthy, don't let xpadneo fool you: In /sys/module/hid_xpadneo/drivers/hid:xpadneo/0005:045E:0B05.0010/report_descriptor you are seeing a descriptor which is already fixed up by xpadneo, usually detectable by the missing trailing zero byte. All the controller models ship with a "broken" descriptor that has a trailing nul byte (Linux doesn't really care, just loudly complains about it in dmesg). Xpadneo fixes it. Xpadneo also swaps a few other bytes in the descriptor to repair the funny axes naming. You can see the function for that in the source code. It's the one with the condensed ifs and hex numbers.
For accessing the original, unmodified descriptor, you'd either need a debug kernel (then there's a rdesc somewhere in sysfs related to the device), or you extract it from btmon. It will be transferred like any other data during connect handshaking, and btmon comes before our driver.
ehats
commented
4 years ago I'll try to go after btmon today. I gave it a quick try yesterday, but I didn't see of that volume when connecting the device. Just to be clear: is this being transferred every time I connect (so sometime between turning the controller on and the "ready-to-go rumble") or only when re-pairing?
kakra
commented
4 years ago Indeed... I just tried myself and I'm no longer seeing the descriptor packet transferred. This has changed. Maybe it explains why the Bluetooth connect process has become more stable.
There seems to be a service record cache in /var/lib/bluetooth/HCI-MAC/cache. If you delete that (while the controller is turned off), it should re-transfer it the next time.
Guessing from the size of the service record, it should contain the HID descriptor.
ehats
commented
4 years ago I stopped the bluetooth service, deleted the entire cache folder, long-pressed the guide button to reset the gamepad, reenabled bluetooth and got the following: this file contains the traffic starting from pairing, then connecting, all the way until the rumble happened. It certainly is a lot of data, but I'm not sure what exactly I'm looking for here :/
kakra
commented
4 years ago Hey what is this magic? ;-)
Still no HID descriptor in the dump. Maybe btmon just does not log it? I'm not sure...
I'll add some code to dump the original HID report descriptor to dmesg... That is, unless you find it in /sys/debug/hid/DEVICE/rdesc. I can't confirm, I currently don't have debugfs enabled.
ehats
commented
4 years ago How about we just look at the "fixed" version and the dmesg fixup messages and revert those bytes back to what they were? I didn't see any ||s in the code, so the original values should be obvious, unless I'm missing something?
kakra
commented
4 years ago This is what I would have done with your logs... ;-) But you got so involved so I thought you'd like to learn a little bit more about the internals. :-)
ehats
commented
4 years ago Haha, I've certainly learnt a lot already (thanks for that!), but I'm also trying to balance it with the amount of time on my hands :)
So, here's the original descriptor then, based on xpadneo's output, manually reverting all of the fixes (except the trailing zero as you said that's not supposed to be there anyway). I also ran it through that online parser tool.
kakra
commented
4 years ago I was thinking about this... It actually doesn't matter much if the report is already partly fixed up. There's just one rule: Modify xpadneo to fixup the report descriptor in a way that the final result exposes a mapping compatible to Windows.
The following document will be helpful for creating the report descriptor patches we are doing in xpadneo_report_fixup(): https://www.usb.org/sites/default/files/documents/hid1_11.pdf (starting at page 24), it explains the byte format and how to walk the descriptor.
Then there's https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf which describes the values (e.g., page 26).
So we'd need to first patch out the ABS_RUDDER and ABS_MISC axes. I'm not sure if we could just patch values BA and whatever ABS_MISC is into 00 to make it "undefined". That way, Linux should map no input event to it and ignore these values if the reports come in. Also, the buttons count needs to be fixed.
In xpadneo_raw_event() we receive the raw events (the HID reports): Here we can intercept the report as received from the controller and modify it before further processing by Linux. I used that to shift the bits for the buttons around to reorder them into the expected order, and also to match it with the Windows mode. There's a heuristic applied: If the report size is 17 bytes, it's a XB1S Windows mode report, if it's 39 bytes, it's a XB1S Linux mode report (that's because the Linux mode HID report descriptor defines a lot of blind buttons and blind axes and also some duplicate items which makes the report much bigger).
As far as I can tell, the HID reports from your XBE2 controller also have a specific size that's neither 17 nor 39 bytes. So it should be easy to detect those and fix those, too. But you don't need to fix values which you excluded from the initial report descriptor anyways (via xpadneo_report_fixup()), so the first round of fixes should be done there.
Now you may ask "but why is it matching on both 17 or 39 bytes length with the same fixups?" That's a good question. The answer is: Because the controller not only has a Linux and a Windows mode but it may actually connect to Linux in a, hmm, let's call it "DID mode" (Dissociative identity disorder): It announces to be in Windows mode but it actually sends Linux mapped reports, or vice versa... This happens when you connected it to the original dongle previously (either using Windows or using the xow project) or to a Windows Bluetooth stack and then re-connect it to Linux without removing it from and adding it again to the Bluetooth stack. Actually, this may partially be a result of the BT service cache we discovered earlier, now that I think about this again.
Whatever it is, the final result should be that the Linux mode of the controller matches the Windows mode when Linux finally sees the packets, because only then it's compatible with Wine games without relying on SDL fixups (which were incomplete when we initially started the driver). The report size itself doesn't matter, just the order of axes has to match, and the order of buttons bits has to match - as long as you're not expecting any applications using hidraw mode. But at least for XB1S, even that should work.
So if you want to try yourself: Try to patch the HID report descriptor (xpadneo_report_fixup()) first, then see what jstest and evtest report, ignore that it may send messed up axes and button values - only the naming and order is important at this point. If that works, adjust the code to patch the HID reports itself (xpadneo_raw_event()) until the values match up with the axes and buttons reported. Finally, add the additional axes and buttons that your controller has in profile 0. Only then I would look at how the different profiles change behavior and how to adjust the code for that. Be careful when putting debug prints into the code as the kernel repeats parsing the events multiple times a blows dmesg up to points where it may scroll multiple seconds for just one button press (that's why I removed those from the original code, as spooling all this to syslog hurts system performance).
Windows compatible mapping and order:
# jstest /dev/input/js0
Driver version is 2.1.0.
Joystick (Xbox Wireless Controller) has 8 axes (X, Y, Z, Rx, Ry, Rz, Hat0X, Hat0Y)
and 10 buttons (BtnA, BtnB, BtnX, BtnY, BtnTL, BtnTR, BtnSelect, BtnStart, BtnThumbL, BtnThumbR).
Testing ... (interrupt to exit)All additional axes should come after those above - which, BTW, is a problem with BTN_MODE as Linux always orders this before BtnThumbL and BtnThumbR and that confuses games. I don't yet know why because it worked with the previous controller firmware. (X,Y) is the left stick, (Rx,Ry) the right stick, (Z,Rz) are the triggers. It should be order (X,Y,Rx,Ry,Z,Rz) in jstest but Linux reorders that. That explains your initial problem. I will be bothering with that in xpadneo 0.9 development, it may be possible to expose the triggers as Accel and Brake to reorder them in joydev without confusing SDL (as that seems to count axes only without looking at the name) but I'm not sure for Wine. The only problem would be that Accel would be left of Brake in joydev which could be counter-intuitive for car drivers. ;-)
ehats
commented
4 years ago Cool, thanks! I'll start hacking around today! In terms of the first PR, would our goal be to have the full reordering / deduplicating done so that evtest/jstest shows something that we can work with?
Also, I've never connected this to Windows device (nor do I have one), or anything else besides two Linux laptops. And I've only ever used USB and Bluetooth, I don't have the dongle.
kakra
commented
4 years ago You can always mark a PR as draft, so we can finalize it over time. If you're unsure how to continue, just push your commits and ask for review.
kakra
commented
4 years ago I recommend having a kernel with debugfs enabled and mounted, you find it at /sys/kernel/debug by default, and within it is hid/DEVICE-ID/rdesc. The latter is a plain text file which dumps the decoded HID report descriptor after patching, so you can verify your changes there. It also dumps all the mappings found. There's also a file events which you can simply cat and see the incoming reports.
ehats
commented
4 years ago Quick question: in the fixup code, you identify devices via if (*rsize >= 81) {. Where did 81 come from? *rsize for me is 1225 (at the start of xpadneo_report_fixup).
I guess we may not want those fixes to be applied to the XBE2, so I'd either need to undo those changes or figure out a better way to identify the models.
ehats
commented
4 years ago Also, is there a way to apply my changes without making new commits every time?
Getting You have already installed v0.7-38-gb632c33-dirty! from ./update.sh. I removed the version check from update.sh, but was wondering if I'm missing something
kakra
commented
4 years ago Quick question: in the fixup code, you identify devices via
if (*rsize >= 81) {. Where did 81 come from?*rsizefor me is 1225 (at the start ofxpadneo_report_fixup).
No, this is not what it does. This is just a sanity check / guard check so that the array access would not be out of bounds. It's directly related to the if (rdesc[##] values following it.
So what it does: It checks if the rdesc is long enough to peek into those rdesc bytes, then it compares if the expected values are at this position and only then patches it. So it's not actually looking at the model but instead if any model has the expected values at this position. So it matches all models that need patching at this point.
I guess we may not want those fixes to be applied to the XBE2, so I'd either need to undo those changes or figure out a better way to identify the models.
See above, it doesn't matter. All we should care about is if we expect some values at this position, and then replace them with something else - guarded by an initial check that the rsize is actually long enough to safely apply this patch.
Also, is there a way to apply my changes without making new commits every time? Getting
You have already installed v0.7-38-gb632c33-dirty!from./update.sh. I removed the version check fromupdate.sh, but was wondering if I'm missing something
Yes, as long as you have a configured kernel source in /usr/src/linux that also matches your current version, you can cd hid-xpadneo (in the xpadneo git dir) and run make modules && sudo make modules_install. I've made a make reinstall which automatically does the former and also reload the module into the kernel. I'm using this for development in Gentoo but Gentoo naturally has a configured kernel source in /usr/src/linux. Not sure for your distribution. You may need to adjust the Makefile to use /lib/modules/X.Y.../build instead, it should be a symlink to a configured kernel source (otherwise I don't know how DKMS should work). You may need to remove xpadneo from DKMS during development, tho, because it may be installed into a different folder and lookup order may conflict. You'll know that you have to do it when the kernel still loads the DKMS module instead of your development build.
kakra
commented
4 years ago Also, you can use git commit --amend to update your latest commit instead of creating a new one.
kakra
commented
4 years ago Let's take this example:
if (*rsize >= 81) {
...
if (rdesc[52] == 0x05 && rdesc[53] == 0x02 &&
rdesc[54] == 0x09 && rdesc[55] == 0xC5) {
hid_notice(hdev, "fixing up Z axis\n");
rdesc[53] = 0x01; /* Simulation -> Gendesk */
rdesc[55] = 0x32; /* Brake -> Z */
}The rdesc size check above it will guard that the following code only patches this context, if all patches would fit into the rdesc.
Then it's comparing that we find Usage Page (Simulation) = 0x05 0x02 and Usage (Brake) = 0x09 0xC5 at this point. Brake belongs into the Simulation page/category (see PDF) but Z is from Gendesk, so we need to patch both values. The result is that we end up with Usage Page (Gendesk) = 0x05 0x01 and Usage (Z) = 0x09 0x32 in the HID descriptor. I'm sure you see that 0x05 means Usage Page and 0x09 means Usage for usage types while 0x32 or 0xC5 are actual codes naming the axes.
From now on, Linux will parse values in HID reports as Z and no longer as Brake.
ehats
commented
4 years ago Yup, I'm brand new to the HID internals, so it's quite a bit to go through at once, but thanks to the guidance it's coming together :) I'm currently going through my descriptor and trying to convince myself if the Usage values are correct or not.
kakra
commented
4 years ago Yup, I'm brand new to the HID internals, so it's quite a bit to go through at once, but thanks to the guidance it's coming together :) I'm currently going through my descriptor and trying to convince myself if the
Usagevalues are correct or not.
Yes, it took me some days to get a native feeling for that. It's quite messy to begin with. When you used the online decoder, it is helpful that it puts the byte values in the first columns, you can look them up in the PDF, then lookup the byte values for the wanted usage type and usage code, and replace them with a patch. If you decode the HID report descriptor again to verify, it now should show the new usage at that line.
Maybe first try to patch values directly in the text field of the online decoder to see and verify the results, then convert them to source code.
Just keep in mind that it's difficult to insert or remove bytes. Replacing bytes is the only easy way to go, so every patch should try to work with only replace instructions.
ehats
commented
4 years ago So I think there's one small piece missing: how do I know what usage I want?
Assuming that the axes appear in jstest follows the order in the descriptor, I got:
but then I have a Usage Page (Sim Ctrls), which seems to be simply a logical grouping, but next up is Usage (Brake). I can see that the code is patching this up, but is this supposed to be axis 4 in jstest? Because if so, this is the weird "D-pad angle" axis on my controller.
Is there a way for me to see to see the axis-usage pairing somewhere or is it just in order of appearance in the descriptor?
kakra
commented
4 years ago Yes, that would be axis 4... But be careful with jstest... It orders axes by Linux HID index number not by order of appearance in the HID report. So after patching, the order in jstest may actually change (while it is still the same in the HID report). To know the order, look here:
https://elixir.bootlin.com/linux/latest/source/include/uapi/linux/input-event-codes.h
jstest simply orders all axis and buttons according to these codes.
But HID reports have a fixed order defined by hardware, and the HID descriptor tells the offsets and lengths of those bits and bytes in the reports.
So, if you rename Brake to Z in the descriptor it will still map to the exact same value in the report, but in jstest the axis will move from the last position to directly after Y (moving the value with it, so it's still reporting the same HID report value although orders do not match). Thus, your original axis 4 in jstest actually just advanced one position forward. jscal -u can be used to define your own order tho I wouldn't recommend to do this now, it should be the final step.
kakra
commented
4 years ago The original Linux mode descriptor of the controller defines the HID report to have (X,Y,Z,Rz), this would exactly match joydev axis 0,1,2,3 in jstest. This looks fine and is actually how things should work until you start a game that uses evdev, and that expects the axes to be named (X,Y,Rx,Ry) - but (Rx,Ry) isn't even defined in the descriptor so the game cannot read the values. That's why I renamed the axes in the descriptor. This makes almost everyone happy and compatible, except joydev which now shuffled axis around thanks to the indexing order.
kakra
commented
4 years ago Since you can easily use jscal -u to define the joydev order without affecting the axis names in evdev, I believe this is the best compromise to meet expectations of all software... That is, if there wasn't Chrome which does it's own special thing and calculates the axes numbers from evdev but reads the values from joydev.
kakra
commented
4 years ago And that is unless we are passed through SDL which has a fix for the controller in Linux mode in later versions but not in earlier versions. So we pretend to be in Windows mode, and then SDL will apply no remapping - no matter which SDL version.
ehats
commented
4 years ago Alright, so to get the axes into a nicer state, I think I'd like to remove two axes for now:
Axis 6 is ABS_RUDDER and axis 9 is ABS_MISC according to jstest, so I was going to replace the usage values with 00 (undefined), but I'm not sure how to find the right bytes in my descriptor. I couldn't find the word rudder anywhere in the descriptor, neither its value BA (from the PDF you linked), with the exception of a single occurrence: 0x0A, 0xBA, 0x00, // Usage (Select Disc), which seems irrelevant.
kakra
commented
4 years ago Could you post the decoded descriptor in its current patched form? It should be easy to identify the lines...
ehats
commented
4 years ago Here you go!
ehats
commented
4 years ago Okay, I think I found the "rudder", as it's the only thing on the device that has a minimum value of 1 (even though its current value is 0 :joy: )
0x05, 0x01, // Usage Page (Generic Desktop Ctrls)
0x09, 0x37, // Usage (Dial)
0x15, 0x01, // Logical Minimum (1)
0x25, 0x08, // Logical Maximum (8)
0x35, 0x00, // Physical Minimum (0)
0x46, 0x3B, 0x01, // Physical Maximum (315)
0x66, 0x14, 0x00, // Unit (System: English Rotation, Length: Centimeter)
0x75, 0x04, // Report Size (4)
0x95, 0x01, // Report Count (1)Is it enough to just go from Dial to Undefined?
kakra
commented
4 years ago Ah, yes that is allowed according to HID: This is for inputs that have an undefined null state... Hat switches use that: They have 4 pressed directions but those have no paired unpressed state: the center is just unpressed for all, and thus an undefined null state as it does not clearly map to any of the directions.
For finding the bytes, you have to move through the descriptor and count bytes: a consecutive sum of report-size=8 makes one byte:
0x05, 0x01, // Usage Page (Generic Desktop Ctrls)
0x09, 0x05, // Usage (Game Pad)
0xA1, 0x01, // Collection (Application)
0x85, 0x01, // Report ID (1)
0x09, 0x01, // Usage (Pointer)
0xA1, 0x00, // Collection (Physical)
// Axis 0, report offset 0+1
0x09, 0x30, // Usage (X)
// Axis 1, report offset 2+3
0x09, 0x31, // Usage (Y)
0x15, 0x00, // Logical Minimum (0)
0x27, 0xFF, 0xFF, 0x00, 0x00, // Logical Maximum (65534)
0x95, 0x02, // Report Count (2)
0x75, 0x10, // Report Size (16)
0x81, 0x02, // Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
0xC0, // End Collection
0x09, 0x01, // Usage (Pointer)
0xA1, 0x00, // Collection (Physical)
// Axis 3 (joydev), report offset 4+5
0x09, 0x33, // Usage (Rx)
// Axis 4 (joydev), report offset 6+7
0x09, 0x34, // Usage (Ry)
0x15, 0x00, // Logical Minimum (0)
0x27, 0xFF, 0xFF, 0x00, 0x00, // Logical Maximum (65534)
0x95, 0x02, // Report Count (2)
0x75, 0x10, // Report Size (16)
0x81, 0x02, // Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
0xC0, // End Collection
0x05, 0x01, // Usage Page (Generic Desktop Ctrls)
// Axis 2 (joydev), report offset 8+9
0x09, 0x32, // Usage (Z)
0x15, 0x00, // Logical Minimum (0)
0x26, 0xFF, 0x03, // Logical Maximum (1023)
0x95, 0x01, // Report Count (1)
0x75, 0x0A, // Report Size (10)
0x81, 0x02, // Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
0x15, 0x00, // Logical Minimum (0)
0x25, 0x00, // Logical Maximum (0)
0x75, 0x06, // Report Size (6)
0x95, 0x01, // Report Count (1)
0x81, 0x03, // Input (Const,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
0x05, 0x01, // Usage Page (Generic Desktop Ctrls)
// Axis 5 (joydev), report offset 10+11
0x09, 0x35, // Usage (Rz)
0x15, 0x00, // Logical Minimum (0)
0x26, 0xFF, 0x03, // Logical Maximum (1023)
0x95, 0x01, // Report Count (1)
0x75, 0x0A, // Report Size (10)
0x81, 0x02, // Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
0x15, 0x00, // Logical Minimum (0)
0x25, 0x00, // Logical Maximum (0)
0x75, 0x06, // Report Size (6)
0x95, 0x01, // Report Count (1)
0x81, 0x03, // Input (Const,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
0x05, 0x01, // Usage Page (Generic Desktop Ctrls)
// Axis 7/8? should be Hat0X,Hat0Y (joydev), report offset 12
0x09, 0x39, // Usage (Hat switch)
0x15, 0x01, // Logical Minimum (1)
0x25, 0x08, // Logical Maximum (8)
0x35, 0x00, // Physical Minimum (0)
0x46, 0x3B, 0x01, // Physical Maximum (315)
0x66, 0x14, 0x00, // Unit (System: English Rotation, Length: Centimeter)
0x75, 0x04, // Report Size (4)
0x95, 0x01, // Report Count (1)
0x81, 0x42, // Input (Data,Var,Abs,No Wrap,Linear,Preferred State,Null State)
0x75, 0x04, // Report Size (4)
0x95, 0x01, // Report Count (1)
0x15, 0x00, // Logical Minimum (0)
0x25, 0x00, // Logical Maximum (0)
0x35, 0x00, // Physical Minimum (0)
0x45, 0x00, // Physical Maximum (0)
0x65, 0x00, // Unit (None)
0x81, 0x03, // Input (Const,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
// Buttons, report offset 13+14
0x05, 0x09, // Usage Page (Button)
0x19, 0x01, // Usage Minimum (0x01)
0x29, 0x0F, // Usage Maximum (0x0F)
0x15, 0x00, // Logical Minimum (0)
0x25, 0x01, // Logical Maximum (1)
0x75, 0x01, // Report Size (1)
0x95, 0x0F, // Report Count (15)
0x81, 0x02, // Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
0x15, 0x00, // Logical Minimum (0)
0x25, 0x00, // Logical Maximum (0)
0x75, 0x01, // Report Size (1)
0x95, 0x01, // Report Count (1)
0x81, 0x03, // Input (Const,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
0x05, 0x0C, // Usage Page (Consumer)
// duplicate button, report offset 15
0x0A, 0x24, 0x02, // Usage (AC Back)
0x15, 0x00, // Logical Minimum (0)
0x25, 0x01, // Logical Maximum (1)
0x95, 0x01, // Report Count (1)
0x75, 0x01, // Report Size (1)
0x81, 0x02, // Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
0x15, 0x00, // Logical Minimum (0)
0x25, 0x00, // Logical Maximum (0)
0x75, 0x07, // Report Size (7)
0x95, 0x01, // Report Count (1)
0x81, 0x03, // Input (Const,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
0x05, 0x01, // Usage Page (Generic Desktop Ctrls)
0x09, 0x01, // Usage (Pointer)
0xA1, 0x00, // Collection (Physical)
// unknown, report offset 16+17
0x09, 0x40, // Usage (Vx)
// unknown, report offset 18+19
0x09, 0x41, // Usage (Vy)
0x15, 0x00, // Logical Minimum (0)
0x27, 0xFF, 0xFF, 0x00, 0x00, // Logical Maximum (65534)
0x95, 0x02, // Report Count (2)
0x75, 0x10, // Report Size (16)
0x81, 0x02, // Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No Null Position)
0xC0, // End CollectionThen look at the HID reports generated and find the bytes modified, with the previous work you can map that back to the descriptor.
The location you've found:
0x05, 0x01, // Usage Page (Generic Desktop Ctrls)
0x09, 0x37, // Usage (Dial)
0x15, 0x01, // Logical Minimum (1)
0x25, 0x08, // Logical Maximum (8)
0x35, 0x00, // Physical Minimum (0)
0x46, 0x3B, 0x01, // Physical Maximum (315)
0x66, 0x14, 0x00, // Unit (System: English Rotation, Length: Centimeter)
0x75, 0x04, // Report Size (4)
0x95, 0x01, // Report Count (1)I'm not sure if this is right: I didn't patch this out and it maps exactly to (Hat0X,Hat0Y) for me and no additional third axis that shows an angle value. Please look in the HID reports: It should update another offset in the report, not just those 4 bits. That second location would be the rudder.
kakra
commented
4 years ago You can also look at /sys/kernel/debug/hid/0005:.../rdesc: It should show you the Linux ABS names including even the HID report byte offsets ("Report Offset" is the byte offset in bits):
INPUT(1)[INPUT]
Field(0)
Physical(GenericDesktop.Pointer)
Application(GenericDesktop.GamePad)
Usage(2)
GenericDesktop.X
GenericDesktop.Y
Logical Minimum(0)
Logical Maximum(65535)
Report Size(16)
Report Count(2)
Report Offset(0)
Flags( Variable Absolute )
Field(1)
Physical(GenericDesktop.Pointer)
Application(GenericDesktop.GamePad)
Usage(2)
GenericDesktop.Rx
GenericDesktop.Ry
Logical Minimum(0)
Logical Maximum(65535)
Report Size(16)
Report Count(2)
Report Offset(32)
Flags( Variable Absolute )Then at the end:
GenericDesktop.X ---> Absolute.X
GenericDesktop.Y ---> Absolute.Y
GenericDesktop.Rx ---> Absolute.Rx
GenericDesktop.Ry ---> Absolute.Ry
GenericDesktop.Z ---> Absolute.Z
GenericDesktop.Rz ---> Absolute.Rz
GenericDesktop.HatSwitch ---> Absolute.Hat0X
Button.0001 ---> Key.BtnA
Button.0002 ---> Key.BtnB
Button.0003 ---> Key.BtnX
Button.0004 ---> Key.BtnY
Button.0005 ---> Key.BtnTL
Button.0006 ---> Key.BtnTR
Button.0007 ---> Key.BtnSelect
Button.0008 ---> Key.BtnStart
Button.0009 ---> Key.BtnThumbL
Button.000a ---> Key.BtnThumbR
Button.000b ---> Key.HomePageIf you cannot identify the axes in the report (maybe it is synthesized from some other value through some Linux HID magic), there's always an option to use USAGE_IGN somewhere in the beginning of the xpadneo code.
But let's always try to follow a rule of fixing things in lower layers before fixing them in the higher layers: descriptor -> raw report -> event (USAGE_IGN, input_mapping and friends) -> user space
ehats
commented
4 years ago I'm not sure if this is right: I didn't patch this out and it maps exactly to (Hat0X,Hat0Y) for me and no additional third axis that shows an angle value.
So I looked a bit more at btmon and jstest. This is indeed the only field that changes when playing around with the D-pad. However! It seems to represent three axes: axes 7 and 8 are HAT0X and HAT0Y as you'd expect, but it's also moving axis 6 (the rudder)
So it looks like the descriptor is correct, but something is interpreting it as three axes instead of two.
kakra
commented
4 years ago So it looks like the descriptor is correct, but something is interpreting it as three axes instead of two.
Then you probably just need to unmap that with USAGE_IGN. Look at evtest to get a clue which code it sends.
kakra
commented
4 years ago If you want to put debug prints, this happens here: xpadneo_input_mapping() - during driver init, this function will get all the codes that Linux fetches from the descriptor. It then walks the array to find a match, and then it returns a value that says either "ignore this", "remap this" or "do whatever you think automatically".
kakra
commented
4 years ago If there's no event for it when passed through mapping, you cannot ignore it. Then it's probably an axis that is made up by joydev.c (kernel) itself and we should look at why and when it does it to work around it.
ehats
commented
4 years ago Alright, I'll give that a shot, but:
/* disable duplicate button */
USAGE_IGN(0xC0224),where did that number come from?
kakra
commented
4 years ago where did that number come from?
From here:
0x0A, 0x24, 0x02, // Usage (AC Back)Usage Page (Consumer) = 0xC (upper 16 bits) + Usage (AC Back) = 0x24 0x02 (in LSF order, so 0x224)
This won't make it easier for you, tho, as it doesn't seem to come from the descriptor.
Please try evtest, it will show you something like this:
Supported events:
Event type 0 (EV_SYN)
Event type 1 (EV_KEY)
Event code 172 (KEY_HOMEPAGE)
Event code 304 (BTN_SOUTH)
Event code 305 (BTN_EAST)
Event code 307 (BTN_NORTH)
Event code 308 (BTN_WEST)
Event code 310 (BTN_TL)
Event code 311 (BTN_TR)
Event code 314 (BTN_SELECT)
Event code 315 (BTN_START)
Event code 317 (BTN_THUMBL)
Event code 318 (BTN_THUMBR)
Event type 3 (EV_ABS)
Event code 0 (ABS_X)
Value 34777
Min 0
Max 65535
Fuzz 255
Flat 4095Event code usually goes to the lower 16 bits with some offset, the upper 16 bits may be from Usage (Simulation).
But I guess it's easier to just put a debug print in xpadneo_mapping() (your version may look slightly different):
#define xpadneo_map_usage_clear(ev) hid_map_usage_clear(hi, usage, bit, max, ev.event_type, (ev).input_code)
static int xpadneo_input_mapping(struct hid_device *hdev, struct hid_input *hi,
struct hid_field *field,
struct hid_usage *usage, unsigned long **bit, int *max)
{
int i = 0;
if (usage->hid == HID_DC_BATTERYSTRENGTH) {
xpadneo_setup_battery(hdev, field);
return MAP_IGNORE;
}
for (i = 0; i < ARRAY_SIZE(xpadneo_usage_maps); i++) {
const struct usage_map *entry = &xpadneo_usage_maps[i];
if (entry->usage == usage->hid) {
if (entry->behaviour == MAP_STATIC)
xpadneo_map_usage_clear(entry->ev);
return entry->behaviour;
}
}
//DEBUG HERE:
hid_info(hdev, "MAP_AUTO: %lx\n", usage->hid);
/* let HID handle this */
return MAP_AUTO;
}I'll need a tiny bit more handholding: I added that debug line, which spewed out around a hundred lines in dmesg, but I don't really know which one to run with. This is what I see in evtest when I press a direction on the d-pad:
Event: time 1593616151.616621, type 3 (EV_ABS), code 16 (ABS_HAT0X), value -1
Event: time 1593616151.616621, type 3 (EV_ABS), code 7 (ABS_RUDDER), value 7You said LSF and that the event code would be in the lower 16 bits, so I looked at the MAP_AUTO output, looking for things that begin with 7, but there's a lot of those.
In the device summary, evtest just says
Event code 7 (ABS_RUDDER)
Value 7
Min 1
Max 8Where's the missing piece that lets me identify the number I need to pass to MAP_IGNORE?
kakra
commented
4 years ago Could you post the hundred lines from dmesg? Maybe I have a clue... (I looked a lot at such numbers during code-redesign)
kakra
commented
4 years ago Also, could you cat /sys/kernel/debug/hid/DEVICE-ID/events and look at that? You need to be root, and it needs debugfs.
ehats
commented
4 years ago here you go. I included a few lines before and after to capture the full flow.
I don't have a kernel set up for debugging yet, so that would need to wait (I've yet to figure out if it's just adding a kernel parameter, or building the whole kernel myself) - let me know if we can't proceed without that and I'll try to set that up too
Describe the bug
When connecting the Xbox Elite Series 2 Wireless via Bluetooth, the mappings are wrong. jstest-gtk reports 10 axes and 26 buttons to begin with, many of them acting "odd". Plugging it in via USB yields 8 axes and 11 buttons.
Here are the default mappings according to jstest-gtk:
Pressing A activates buttons 0 and 10. B activates 1 and 12. X activates 3 and 14. Y activates 4 and 15. SELECT doesn't work. MENU activates 22. Using the D-pad affects axes 7 and 8 as expected, but it also influences axis 6 in a seemingly random way.
Steps to Reproduce
Connect via USB, wait for the rumble and the above happens every time.
Expected behavior
Buttons and axes to be mapped correctly.
System information
Controller and Bluetooth information
No dongle, it's a
Intel Corporation Wireless 8265 / 8275 (rev 78)in my laptop. Also tested with a 9560 and the results were the same.xpadneo-dmesg.txt xpadneo-btmon.txt