spec vibrationActuator gamepad extension

[nondebug]

Add a single vibrationActuator member to handle haptic effects that affect the whole gamepad.

[luser]

This all looks pretty sensible for handling the common case of a gamepad with weak+strong rumble motors, and it seems like it'll provide some extension room for handling more interesting things. I don't know of a lot of other haptic designs in the wild aside from:

• The Xbox One controller, which has the usual dual rumble motors + two more "impulse triggers", one near each trigger button. In the past I hadn't seen any API that actually exposed them (since I don't have access to the Xbox One SDK), but it looks like Microsoft exposes them via the WinRT Gamepad APIs nowadays.
• The Nintendo Switch Joy-Cons, which have "HD rumble", which is apparently provided by a linear resonant actuator and the API involves loading waveforms representing the haptic (an LRA is functionally very similar to an audio speaker).
• The Steam Controller also has LRAs, one under each touchpad, but they seem to be dual-axis actuators. (The iFixit teardown says the touchpads are Cirque GlidePoint TM040040s, and the Cirque page mentions that those are compatible with Alps HAPTIC Reactor drivers, which look like what's in the photo.) I'm not familiar with the Steam Controller API, but the API contains methods like TriggerHapticPulse, which seems to offer only a simplistic rumble-like interface to the LRAs. There's also a TriggerVibration which is a very simple interface that simply lets you turn on the left and right rumble motors for a controller for a period of time.
• Old-style force feedback joysticks, which had super complicated APIs for creating and playing effects. I'm not sure if controllers like this are even manufactured anymore.

In any event, most of those seem like they could be handled with future extensions to this API. The only thing I'd worry about is that if code in the wild adds support for dual-rumble haptics but then the spec gets extended to add something more complex (even just the 4 rumble motors in the Xbox One, say) there doesn't seem to be a fallback path so that existing code can work with newer devices.

What if instead of having readonly attribute GamepadHapticActuatorType type on GamepadHapticActuator we instead had something like bool canPlayEffectType(GamepadHapticEffectType type) or a WebIDL setlike type that gave the set of GamepadHapticActuatorType or GamepadHapticEffectType values that were supported? With an interface like that, users could detect if a device supports dual-rumble and then use it, and if support for a more complex haptic is added to the spec and implemented by browsers, browsers could have the device support both the newer type and the dual-rumble type, so existing code would work with newer devices, and new code could feature-detect for quad-rumble or whatever if it wanted to support that.

[qdot]

Old-style force feedback joysticks, which had super complicated APIs for creating and playing effects. I'm not sure if controllers like this are even manufactured anymore.

I'm not sure if flight and wheel joysticks still use these, as most of those use USB so latency isn't as much of an issue as it used to be. However, I think we may see a return to this with HD haptics, since the timing can be precise enough that having parametric effects might be useful. I think that may be beyond the scope of "gamepads", but with the way gamepads are being repurposed for VR and including other sensors, it's kinda hard to tell?

The Nintendo Switch Joy-Cons, which have "HD rumble"

The Nintendo Switch API seems to expose "amplitude" and "frequency" versus a simple "low/high power level" like XInput. More info can be found at:

dekuNukem/Nintendo_Switch_Reverse_Engineering#7 dekuNukem/Nintendo_Switch_Reverse_Engineering#11 https://github.com/dekuNukem/Nintendo_Switch_Reverse_Engineering/blob/master/rumble_data_table.md

And there's Haptics/Protocol code (C++) at:

https://github.com/mfosse/JoyCon-Driver/blob/master/joycon-driver/include/Joycon.hpp

I could see that getting sort of "dumbed down" the same way the Steam API does it though, which would basically wedge it into working under the proposed API.

Returning to the spec, the use of "HapticActuator" versus "VibrationActuator" is a little confusing. Depending on how much future-proofing needs to happen in the spec, is it useful to make sure these terms are divided up correctly? Vibration is only one type of actuation, but this doesn't really take other mechanisms like temperature, pneumatics, etc into account, which could show up in the future. Not sure if that really needs to be accounted for though, and definitely don't want to turn this bug into bikeshedding around that.

[nondebug]

Thanks for the comments!

I agree, a canPlayEffectType method would work well and would allow the actuator type to be removed. I can definitely see the actuator type becoming an issue, especially with actuators that can reasonably be classified as multiple types. Done.

Regarding haptic vs vibration naming, I chose vibrationActuator for the Gamepad member name because it should be limited only to effects that vibrate the whole device. Rumble is common enough that I think it's worth having an explicit member to signal the capability rather than hiding it in a hapticActuators array. But, I expect we will eventually also want an array to cover other types of vibration and other types of actuators. I left it out only because it's not useful yet and would unnecessarily complicate the spec.

For instance, Xbox One trigger rumble is a vibration effect that independently vibrates one element on the gamepad. I think it would be appropriate to expose these as separate actuators in a hapticActuators array and define a canonical haptics mapping in the Standard Gamepad. (I'd prefer this over adding "quad-rumble") Similarly, if you have dual Joy-Cons or a Switch Pro controller, LRA vibration effects can be independently controlled between the two hands. I'd rather expose left and right GamepadHapticActuators that each control one Joy-Con than add a "stereo-switch-lra" effect type.

In that context, does it make more sense that the vibration-focused actuator would be vibrationActuator, and the more general collection would be hapticActuators? Calling it "vibrationActuator" has one more benefit that I find appealing, "vibration" is more obviously connected to rumble effects so I expect developers will try it first.

This API is likely not sufficient for waveform-based haptics. It should be possible to upload an effect to the device independently of playing the effect. The waveform data might be large and uploading it could incur a delay or use battery power. And because the storage of such effects isn't infinite, it should also be possible to query if an effect will fit and discard effects from the device once they are no longer needed. The use case I think of is a vibration effect based on an audio track. Each effect is a unique segment of the audio and can be discarded immediately after playing, but multiple effects can be buffered in advance for smooth playback with no gaps.

It's probably not sufficient for joysticks with directional haptics that react to the stick's position and velocity, either. I haven't thought about these too much, IMO this joystick-specific feature is not sufficiently gamepad-like to worry about here. It would be better if we could support these legacy devices through Web USB.

[LJWatson]

Is this PR something to defer to v2? Ping @Sagoston and @SGraham

[marcoscaceres]

Fixed merge conflict.