Vibrate feature for Gamepad

[goldenratio]

Similar to navigator.vibrate(), we need vibrate feature for Gamepad API. What do you all think? I should be able to vibrate gamepad, if the gamepad supports vibration (example: XBOX 360)

[cvan]

There was a thread from April 2014 on the public-webapps mailing list about adding something like Gamepad.vibrate.

Also there's a Firefox platform bug on file for implementation (couldn't find a bug in the Chromium tracker). See this comment by @luser in the Firefox bug.

[goldenratio]

ok, I created new issue in Chromium tracker https://code.google.com/p/chromium/issues/detail?id=581515

Seems like no progress from Firefox side.

[cvan]

Also see this mailing list thread started by @toji: https://lists.w3.org/Archives/Public/public-webapps/2016AprJun/0052.html

We ought to probably keep the discussion here now.

[luser]

As with most things, I think we're still primarily blocked on figuring out a sensible API that can expose the features of modern devices without being painful to use. We wanted to try to reuse the Web Vibration API interfaces, but they are not expressive enough for the needs of the Gamepad API. Primarily:

• Multiple vibration motors per device: most modern gamepads have two, the Xbox One controller has four, and generally the motors are of varying strengths. (Usually at least one strong and one weak vibration motor.)
• Vibration motors allow setting the vibration strength, not just the duration.

[dwoldrich]

@luser Thanks for all your work on Gamepad to-date, it's so neat to see most browsers now supporting it.

I have a concern with a Gamepad Vibration/Force Feedback API that the user's JavaScript could send too many change commands too quickly for some devices to make sense of. If HTML5 does get Gamepad Vibration, could there be some sort of error code response to say "you're sending too much data, try again later"? Maybe there could simply be a rule that only one command could be sent per axis per time slice?

That might be too harsh, perhaps throttling the axis update data rate could be event-driven, but in any case, I would like an API that would perform reasonably well on most hardware and operating systems by holding the developer's hand a little and not letting him abuse the device.

[toji]

I think it's totally reasonable to build some sort of basic limits into what you can ask the device to do. For instance: with XInput you trigger vibration by setting an intensity value and the pad will continue to vibrate at that intensity level until you set it to an intensity of zero. That seems totally inappropriate for a web API, as it would lend itself to easy abuse. Something based on sending vibration durations (in addition to intensity) that have reasonable caps seems much more web friendly.

[cvan]

What about introducing the following?

• Gamepad#haptics - an array of haptic devices
• GamepadHaptic - an interface for a haptic motor exposed from the parent Gamepad, à la GamepadButton (could get messy with buttons/axes sometimes dualing as/containing haptic motors)
• GamepadHaptic#rumble(intensity, duration) - a method that allows better granularity than does the Vibration API (currently)

Though valid point about abuse. Personally, I'd prefer to do what browsers have done with sites that abuse window.open or alert - detect/rate limit the pop-up windows or alerts and prompt the user to stop the "abuse."

Does that sit well with folks?

[toji]

This feels pretty good to me! A few thoughts:

• The naming doesn't matter to me too much, but maybe "vibrate" meshes better with existing interfaces than "rumble"? ("Haptics" feels like a great name, though!)
• Keeping with the existing vibration API, it would be nice if the duration accepted both a single duration and an array to use as a pattern.
• Assuming that intensity is a normalized 0-1 float? Should also specify that it may be snapped, so that devices with only a single intensity (Vive controllers) treat 0.8 as 1.0 (not sure if it should be rounded or simply non-zero in that case.)
• For devices like the Xbox pad where the motors are different frequencies I've heard requests that we somehow indicate the type of motor being interacted with. It's an interesting idea, but I don't have a clue how to meaningfully expose such a thing. If you have thoughts on it would be interesting to discuss, but I think it's probably just not realistic.

[dwoldrich]

I only have a tiny dog in this hunt, but it seems to me that we should cover both rumble and force feedback in a haptics API. MIcrosoft's DirectInput API lets you design an effect and upload it into the device and then start it with a subsequent API call. https://msdn.microsoft.com/en-us/library/windows/desktop/ee417563(v=vs.85).aspx

Regarding the abuse issue, I don't think the API would be very friendly if it could throw up a "Page Unresponsive"-like dialog if the JavaScript has crossed some internal spam threshold on the update calls.

I looked at how the Web Audio API handles throttling the audio updates, and they use the onaudioprocess event to signal when a buffer has started to play. The developer can use onaudioprocess to double-buffer their audio buffers and play infinite length raw or generated audio streams.

Would there be any utility in having a "onhapticsprocess" event that could be used to similarly double-buffer updates sent to the rumble motors? Perhaps spammy calls to the haptics API's could be dropped, and only the latest update sent to each motor before onhapticsprocess fires would be processed by the device?

I like the Haptics naming too!

[toji]

I'd need some serious convincing that DirectInput is worth supporting at all at this point. As far as I can tell Microsoft has abandoned it and I don't recall the last time I saw a device that advertised Force Feedback support as a feature. Plus, given how different it is from the basic rumble haptics modern devices support I can't imagine that it's worth designing a new API around accommodating them.

The rest of your comments are :+1: though!

[dwoldrich]

@toji The first time I moved a physical object in the world with my code was using DirectInput with the Microsoft Sidewinder Pro joystick, so I admit to having an enthusiasm for force feedback that the rest of the world does not share. :-(

If we squint, could we consider rumble gamepads to support the default "force feedback 'effect'" which is the rumble plus an intensity setting?

Reading over the Wikipedia article on the DirectInput situation, it sounds like Microsoft confused a lot of folks: https://en.wikipedia.org/wiki/DirectInput#DirectInput_vs_XInput

I don't think Microsoft is necessarily anti-force feedback, they're just not going to iterate on the API for DirectInput due to low consumer demand, and they're going to focus on gamepads with XInput. If you're trying to read from keyboards and mouse, you still need to rely on DirectInput, so DI is not super-duper dead.

Looking at Logitech's Force Feedback steering wheels, it does appear to have DirectInput driver support, but then they layer on their own API atop DirectInput to do extra stuff like flash LED's on the steering wheel to indicate RPM's and such! Pretty neat. Anyhow, that's one example of a contemporary controller that does support force feedback using DirectInput.

It would be good to get some Microsoft and/or FF device company peeps in here to reassure us or wave us off of Force Feedback.

[luser]

I'm not interested in spec'ing force feedback. I'd like to spec simple rumble that covers what current gamepads do: one or more vibration motors of varying intensities.

[dwoldrich]

@luser That's cool, and is probably a wise decision for cross-platform. Having rumble support will be so sweet.

Please consider my event-based update rate throttling concept in your spec to prevent the developer from spamming the device.

[kg]

"Spamming the device" isn't really a meaningful concept for gamepad haptics any more than the player pressing buttons 20 times a second would be spamming. Many game genres will have the haptics running at some intensity (probably low) virtually all the time - car and space sims, for example, to provide force feedback. The XBox One's controller specifically added trigger haptics in order to enable scenarios like this.

What kind of DoS/malware scenarios would be enabled by being able to just drive the haptics? The user already has to opt-in by pressing a button on the controller.

Update rate throttling for haptics also seems misguided. You're just increasing the latency of the haptic response which is going to make it way less satisfying (if not actively confusing).

[dwoldrich]

@kg how blocked is your DOM thread that event-driving a signal to update the motor intensity would add perceptible latency? The fire rate of a JavaScript event can be set to whatever frequency makes sense for the browser vendor/device. Regarding your example, a user pressing buttons is input, rate-limited by the device and speed of human hands. Haptics is output, rate limited by ... the writer of the spec? browser vendors? device drivers?

I hadn't considered DoS was an issue with haptics. Any decent browser/OS/device driver set is going to protect the system against abuse like that. What I'm concerned about is the web developer sending noise to the device because they don't know how to program. By explicitly telling the developer "it's okay to update haptics now" by giving them an event to subscribe to, we can impose some order on them and take any guesswork out of when they should program the thing. Just like outputted raw audio wave data flow control is achieved via the onaudioprocess event, I'm suggesting we would benefit from giving a similar flow control via events to haptics.

Giving developers an event is about helping them organize their thoughts, asking them to pick out the best signal to send to a motor based on what's happened recently in their simulation, etc. If they still want to spam like maniacs, that's fine, we can ignore superfluous signals and they'll get noise on the device; but at least we can point them to the event when they complain that their code doesn't work right.

Automatically triggered haptics coupled to button presses also sounds like a really useful concept to include in the API for those applications that can use it. Please add that too, @luser!

[kg]

If you just mean event-based 'pull' haptics, that sounds totally fine as long as the rate is high enough. I got the impression you meant that the haptics would be 'push' and rate limited to like 10hz.

[Armaldio]

Hey all,

Any updates/ETA for this ? I'm really interested in using this in the browser

Thanks

[patricknelson]

@toji Do you know if this happens to already be exposed somehow in the latest experimental builds of Chrome? I'd love to hack something together, even if it's volatile 🔥 API with the knowledge that it could change at a moment's notice. I can just abstract this away in my own code until the dust settles.

[brmscheiner]

Bump! Our team needs this :)

[Hessamshar]

Any news on this? Working on a web based game and having this feature would be great.

[nondebug]

I created a Chrome Platform Status item to track the progress of this feature in Chrome. On the status page there's a link with some proposed changes to the Gamepad API to support rumble-style effects. Please take a look!

It's similar to the Gamepad Extensions draft but a bit more generic in order to support future extensions. I removed pulse because it is not extensible enough for a generic haptic effect. For the sake of buffered effects (which are not supported but probably will be someday) it is a good idea to separate creation and playback, giving the developer a chance to upload large effect descriptions when it is most convenient.

I also think we should map DualShock-style rumble motors as a single "dual-rumble" actuator rather than two "vibration" actuators so we can program them with a single "dual-rumble" effect. This makes it explicit that the effect's purpose is to vibrate the whole gamepad (as opposed to a single button or axis, eg Xbox One rumble triggers) and enables reasonable fallbacks for gamepads that can vibrate but don't have a standard dual-rumble configuration. I'm thinking of the Steam Controller with its LRAs, gamepads with a single ERM motor, and composite devices like the Xperia PLAY that reuse the pager motor for rumble.

[patrickkettner]

The Edge team reviewed the specification, and we have no objections to it. The closest thing to a concern is that it seems like WebVR was moving away from using gamepad for motioncontrollers in v2. If that is the case, then we will be less likely to implement as quickly.

[scheib]

Thanks @patrickkettner, re WebVR moving away, we have proposals to harmonize: https://github.com/w3c/webvr/issues/298 https://github.com/w3c/webvr/issues/297

[daoshengmu]

In Gamepad extension, we already have readonly attribute FrozenArray<GamepadHapticActuator> hapticActuators; in Gamepad [1]. The interesting part is Chrome also define another GamepadHapticActuator interface that has the same with the one in GamepadExtension[2][3]. I would suggest that we should unify the spec and try to improve of the part in GamepadExtension instead of making a new one. At least, GamepadExtension is a Editor's Draft.

[1] https://w3c.github.io/gamepad/extensions.html#partial-gamepad-interface [2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/blink/renderer/modules/gamepad/gamepad_haptic_actuator.idl;l=26?originalUrl=https:%2F%2Fcs.chromium.org%2F [3] https://w3c.github.io/gamepad/extensions.html#gamepadhapticactuator-interface

[razvanphp]

We are also working on a web based game and we are able to vibrate the xbox 360 controller, but not racing wheels from logitech, fanatec or thrustmaster, even tho they are detected in gamepad API. This test website implemented it also: https://gamepad-tester.com

Is this something browsers/chrome needs to update or it is something related to the drivers? Thanks!

[marcoscaceres]

We will be revisiting this hopefully later this year.

[razvanphp]

@marcoscaceres can we sponsor this development? What would be the best approach to speed things up, since we really need this feature for our game. Thank you!

[nondebug]

@razvanphp I think it would be good to support racing wheels through Gamepad API but so far there isn't enough interest to justify the work.

The main issue is that platform APIs historically haven't supported racing wheels very well so we'd have to put in a lot of effort on the browser side to deal with vendor-specific behavior and quirks. We do this in Chromium for a handful of popular gamepads but it doesn't scale. I'd like to see evidence of better platform-level support for wheels before we add explicit support to the spec.

A second issue is that racing wheels can be pretty dangerous since they use a powerful motor to turn the wheel. The device doesn't usually enforce any sort of safety precautions so it would be up to the browser to make sure the commands sent to the wheel don't harm the device or the user. I'm not sure that this can be done in a generic way, probably we would only want to support haptics on certified devices where we have some assurances about safety.

It's possible that Windows.Gaming.Input already does a good job of supporting racing wheels. I don't have access to all the necessary hardware so I haven't been able to test if WGI works consistently across wheels from different vendors. If you have access to the hardware and are willing to do some investigation, documenting the current level of support would be helpful as a first step toward justifying the API change.

The quickest path to supporting racing wheels in Chromium is through WebHID, but you'll need to craft the haptics output reports yourself. It might be better to do this anyway since even if we add support to the API it's likely that we won't support every feature of every wheel.

It should be possible to create a polyfill (a shim) that uses WebHID to communicate with a racing wheel but reformats the inputs to match the Gamepad API.

[razvanphp]

Thank you for the detailed explanation. Not sure if I made this clear in my first question, but having only vibration support would be already enough, to be able to implement feedback when hitting obstacles, just the way it already works for X360 controller. Is this still a safety issue or vendor specific implementation?

[razvanphp]

@nondebug sorry to insist on this one, but we have no other lead. Would implementing at least the vibration (1 motor) for racing wheels from Logitech & Thrustmaster be rather easily doable, similar to the Xbox controllers? That would cover probably more than 80% of the use-cases already and would give the games a real feel.

Also, would developing/switching to chromium would solve this issue? Should the driver providers implement this? Could you please point us into the direction of where to find developers/code that needed to be changed for this? We would love to sponsor this, as mentioned before.

Thank you! R

[nondebug]

@razvanphp I think this is not easily doable, but patches are welcome and I think this would be a good addition to the API.

Note that Microsoft is now interested in developing a new HapticsDevice API which I think would be the best path forward to support advanced haptics for racing wheels. We can still support wheel vibration through vibrationActuator but HapticsDevice could enable support for more advanced haptic effects. Please take a look at the explainer and consider reaching out to @bmathwig to discuss how racing wheels would be supported.

I purchased a Logitech G29 to investigate adding support for racing wheels and found that it did not have a dedicated vibration actuator. Instead, applications can rapidly turn the wheel actuator back and forth to emulate a rumble effect.

I don't know how we can do this safely without vendor support. If we're sending commands to control wheel haptics we need to be very careful not to send commands that would cause the wheel to rotate with dangerous force. The difference between a "dangerous" haptics command and a "safe" rumble command is subtle enough that I'm not confident we can write a simple test to guarantee we are always operating safely. It needs to be fail-safe so that if the browser exits unexpectedly the device is always released in a safe state. I would rather rely on vendor documentation or drivers to ensure we are operating within safe parameters.

The first step is to create an application demonstrating how to simulate a dual-rumble vibration effect on a specific Logitech or Thrustmaster device. dual-rumble has two vibration channels that rumble at different (unspecified) frequencies, a lower-frequency "strong" channel and a higher-frequency "weak" channel. Ideally we would generate an effect that scales two oscillators and adds them together to represent the "strong" and "weak" actuators. I'm not sure whether wheel actuators are good enough to simulate that sort of combined effect. If that's not achievable then we could simply use max(strong,weak) to scale a single oscillator.

On Windows, I suspect we can support racing wheel haptics through Windows.Gaming.Input.ForceFeedback and this might be a good option for developing the demo app. As an alternative, we have an ongoing effort to add a new Windows data fetcher that consumes inputs from the Windows.Gaming.Input API instead of XInput: WgiDataFetcherWin. Support for dual-rumble was recently added. It might be easier to add experimental racing wheel support to WgiDataFetcherWin than to develop a standalone demo. If you're interested in going that route, let's coordinate on the bug.

[marcoscaceres]

Hey folks, we are about to land vibratorActuator into the main spec: https://github.com/w3c/gamepad/pull/190

We would appreciate any last review there.

[razvanphp]

@marcoscaceres thank you for the heads up!

Is the Chrome implementation just for Xbox Controllers? This means that other gamepads like racing wheels still cannot make use of it at this stage.