Mapping of HID Events to the Gamepad API

[hoene]

Most gamepads use the HID protocol to communicate with the computer, either via USB or Bluetooth.

In general, the document specifies on how to support a HID Gamepad. In practise, the support of operating system is much more limited (e.g. http://electronics.stackexchange.com/questions/141883/composite-hid-gamepad-descriptor-works-on-windows-but-not-on-linux)

For the new gamepad extensions, sensor data be transmitted using the "HID Sensor Usage Tables". http://www.usb.org/developers/hidpage/HUTRR39b.pdf

But then again, it is not clear on how HID related to the gamepad extension API.

I would suggest to define a proper mapping fom HID to W3C - other, I guess, we will get a mess.

[techtonik]

Yes. When reading https://w3c.github.io/gamepad/#dom-gamepad-mapping I've got a feeling that the spec doesn't solve the problem of identifying signals from device, but redirects it.

For me, the ideal spec starts with a picture, so current spec is good. Then I would assign an event for every control on a physical device. Reusing table 12 from USB HID is nice, but it needs pictures. Table 12 is about keyboard, so event type is key press and release or activate and deactivate. SDL2 reuses the same constants for key identification, it makes sense to mark Gamepad control the same way - so that keys/controls that users expect to match on different devices produced the same event values. This way the need for mapping can be reduced. Or the mapping could be more intelligent - for rebinding keys/actions to what users want.

So, picture of keyboard, mapping of every physical key to its numeric id - 0x003A being F1 and so on according to USB HID page. Then the same picture for gamepad, which starts with 0x0100 for example. Note that this doesn't specify the value that control returns. It just unifies numeric identifier for controls.

[beidson]

Direct HID event mapping is not available on all platforms that support gamepads, and will not be.

Let's not do this.

[techtonik]

What is direct HID event to understand why device event may not be available? By the way, I am speaking only about control (keys, shutters) identifiers. Events can be described only after their sources can be properly identified.

[luser]

Direct HID event mapping is not available on all platforms that support gamepads, and will not be.

Let's not do this.

I think it would be useful to have this for HID devices, but yes, not all gamepads are HID devices. Microsoft's XInput API and Apple's Game Controller framework both provide non-HID APIs to devices.

For arbitrary HID devices though, it would be great for extensibility (so the spec doesn't have to cover every possible type of device input/sensor/output) as well as a way to ensure consistent behavior across implementations and operating systems--the same device descriptor should result in the same buttons/axes.

[nondebug]

Many gamepads report HID usages from multiple pages, e.g. Consumer page is sometimes used for the Meta and Back keys on Android gamepads. So you would need at least a usage page/usage ID pair for each input.

HID usages aren't used consistently across devices so it's not clear to me how this would be implemented. Also, the Standard Gamepad doesn't try to map buttons by usage. It describes a mapping based on the location of the button on the gamepad and not its intended purpose.

I think the task of mapping from inconsistent HID usages to canonical Standard Gamepad button/axis indices is best left to a library that has knowledge of specific devices and can be updated to support new devices without requiring changes to the browser. Encoding this information in the browser makes the API easier to use, but is hard to scale. To make the button/axis data more easily usable by a library it would be better for the Gamepad spec to provide inputs in a stable ordering, perhaps by preserving the order of appearance in the input report. This might be more appropriate for a Web HID API than the Gamepad API.

[beidson]

(An update on my thoughts here)

Been hacking on WebKit's HID implementation over the last few weeks, digging deep into the HID weeds.

There's simply no way to do this right. HID descriptors are "standardized" and you can match an input report up to the elements in the descriptor... but beyond that...

I have on my desk:

• A controller that reports (4) x-axis, but only updates the 4th one when the x-axis moves
• A controller that reports 10 buttons, with only 8 physically on the device
• A controller that reports 11 buttons, with only 8 physically on the device
• A controller that reports a z-axis, but there's no analog stick or other element on the device that controls its value. In fact, it's value just moves around randomly
• A controller with a 4-bit hat switch d-pad that uses the values 1-8 to report the 8 possible activations, and 9 to report the null position
• A controller with a 4-bit hat switch d-pad that uses the values 10-3 to report the 8 possible activations (opposite direction from the previous controller) and 2 to report the null position
• A controller that reports 19 buttons but only has 17
• That same controller offers analog values for 12 of its buttons through 12 "pointer" elements that don't have any specific usage - just "pointer"

I'm afraid individual controls have individual mapping needs, and there's no way a w3c spec can accommodate that generally speaking.

[luser]

I'm afraid individual controls have individual mapping needs, and there's no way a w3c spec can accommodate that generally speaking.

I fell into this same pit of despair a while back. I think there are two plausible paths here:

1. Spec out an algorithm for taking a HID descriptor and producing a mapping from its elements to the buttons/axes arrays in the Gamepad spec. Even if the resulting Gamepad object doesn't look sensible for every controller, if it is at least consistent across implementations then it ought to be possible to maintain a database of known controller layouts (I prototyped such a thing a while ago) and a small JS library that could produce the "standard" mapping from the consistently-mapped Gamepad.
2. Give up and expose the HID descriptor + reports in the API somehow. With all the knowledge I have now I honestly wish we would have just built a WebHID spec instead of this one, but that ship has sailed. :) WebBluetooth doesn't currently expose HID devices, AIUI, because of the security concerns of allowing web content access to things like your keyboard/mouse, but maybe a simple way to prototype this would be to tweak Chrome's implementation to allow Bluetooth HID devices that report themselves as joysticks/gamepads, and see how that feels in JS?

[nondebug]

@luser: WebHID now exists and exposes USB and Bluetooth HID gamepads: https://wicg.github.io/webhid/

We can create a getGamepads shim that inserts WebHID-backed gamepads to experiment with exposing HID usage info. Unfortunately the necessary usage info isn't exposed on Windows Chrome yet, but on Mac and Linux the HIDDevice.collections member contains all the data parsed from the HID report descriptor, including usage IDs.

1. Spec out an algorithm for taking a HID descriptor and producing a mapping from its elements to the buttons/axes arrays in the Gamepad spec.

As a starting point for discussion I'll describe what we do in Chrome. Button indices are assigned in two passes. We start by assigning indices based on usage IDs in the Button usage page (UP:0009). For instance, the first button usage (UP:0009 U:0001) becomes buttons[0], U:0002 becomes buttons[1], etc. I think this makes the most sense for the majority of HID gamepads. Unfortunately there are gamepads that have non-Button buttons. Chrome inserts these "special" buttons in a second pass using any indices that were unused in the first pass, in order of increasing usage page and usage ID.

Axes are assigned by usage ID, only considering usages from the Generic Desktop page with U:0030 or greater. Generic Desktop X (UP:0001 U:0030) becomes axes[0], Generic Desktop Y (UP:0001 U:0031) becomes axes[1], etc. All axes are assigned in one pass.

Some issues that aren't addressed well by Chrome's implementation:

Hat switches should have special handling. Generic Desktop Hat Switch (UP:0001 U:0039) is almost always a 4-bit rotational value with logical range 0 to 7. When no direction is pressed (null state), it reports the out-of-bounds value 8. If we naively scale to the logical range we get an axis value that violates the spec (axes should be in the range [-1,+1]). But if we clip the value, applications can't detect the null state. Probably we should have special handling for mapping an 8-direction hat switch axis to buttons.

Multiple inputs with the same usage ID. This is uncommon for HID gamepads but the spec allows it. For instance, instead of using X/Y Z/Rz for the left and right thumbstick axes, a device might use X/Y usages for both. A device can also use the same button ID for multiple button inputs, which is more commonly a sign of a misconfigured report descriptor but is also allowed. Chrome only uses the first input it encounters for each usage ID.

"Special" gamepad buttons are defined in a hard-coded list: Generic Desktop System Main Menu (UP:0001 U:0085), used by some Xbox One gamepads on old firmware over BT Consumer Power (UP:000C U:0030), used by the Nvidia Shield 2015 gamepad Consumer Search (UP:000C, U:0221), used by the Nvidia Shield 2017 gamepad Consumer Home (UP:000C, U:0223), "home button" usage on Android-friendly gamepads Consumer Back (UP:000C, U:0224), "back button" usage on Android-friendly gamepads

On one hand, it would be nice to expand this to include anything that might be a "special" HID button. On the other, if any HID device with at least one button counts as a gamepad then we should be careful about including usages that aren't typical gamepad button usages. Maybe a gamepad should have at least one Button button before allowing "special" buttons.