Add support for a PCM buffer to the gamepad actuator

[cabanier]

The Quest Pro controller comes with a new VCM motor for haptics that has a variable frequency and intensity. The OpenXR API is documented here

We'd like to expose this to WebXR developers so they can generate more immersive haptics and would like to propose the following function to the GamepadHapticActuator IDL: Promise<GamepadHapticsResult> playPCM(audioBuffer);

cc @nondebug

[cabanier]

@marcoscaceres what is the best way to move forward on this? A PR on the spec?

[nondebug]

We want to move away from supporting gamepad haptics with GamepadHapticActuator and adopt a new strategy that can work for other (non-gamepad) haptic devices. We'll keep GamepadHapticActuator as a legacy API to support "dual-rumble" haptics but we don't plan to add support for buffered haptics or other new effect types.

The next step is to draft a specification for HapticsDevice API, then create a PR to integrate HapticsDevice into Gamepad API.

[cabanier]

@nondebug that explainer was written 2 years ago and doesn't cover the case where we want to pass any waveform. Will there be more progress on this spec? Who is working on it?

I agree that the current haptics interface isn't great but I don't see a need to completely replace it.

[nondebug]

@bmathwig can you comment on whether this is in scope for HapticsDevice API?

Supporting custom haptic waveforms is explicitly listed in the explainer's goals so I think this is intended to be supported in the API although that part of the spec hasn't been worked on yet.

[cabanier]

Looking at the proposal, HapticsDevice is on PointerEvent. How would that work on gamepad or WebXR?

[nondebug]

PointerEvent integration is for supporting pointing devices with tactile feedback like Microsoft Surface pens. Gamepad integration wouldn't use PointerEvent. Probably we'd have an attribute that returns an array of HapticsDevices representing the actuators on the device.

[marcoscaceres]

I agree with @nondebug. We should figure out the requirements as part of haptics incubation.

@cabanier, I know the explainer hasn't moved in a while, but we should probably move that whole thing to the WICG to have these discussions.

[cabanier]

PointerEvent integration is for supporting pointing devices with tactile feedback like Microsoft Surface pens. Gamepad integration wouldn't use PointerEvent. Probably we'd have an attribute that returns an array of HapticsDevices representing the actuators on the device.

Cool but that sounds a lot like the HapticActuators array we have today (and have to maintain). Why not just extend that?

[cabanier]

@cabanier, I know the explainer hasn't moved in a while, but we should probably move that whole thing to the WICG to have these discussions.

That sounds good. What is a realistic timeframe? Since we're already selling hardware with these capabilities, we'd like to have it as soon as possible.

[bmathwig]

I'm planning on picking up the HapticsDevice API rewrite next month. Microsoft will be doing prototyping of the new API upstream in Chromium during the latter half of this year. The plan is to support PCM buffers as well as expand the scope of devices supported beyond haptics-enabled pens.

[razvanphp]

@bmathwig where can we follow this development?

We are actually following up on #19 to include the racing wheels vibration capabilities for in-browser games, having a more realistic and immersive experience for the users.

[cabanier]

There hasn't been an update from @bmathwig since that last message :-\

[nondebug]

Ben is no longer working on this. I reached out to @SteveBeckerMSFT and @gabrielsanbrito to see if they can pick it up.

@qqharryhwang presented on haptics APIs at TPAC and may also be interested.