WebXR emulator extension

fernandojsg commented 4 years ago

We, at mozilla mixed reality, have been discussing for a while about the need of better tools for prototyping and testing WebXR apps, as currently is quite a pain the need of an actual device to test your code. One of the ideas was about creating a extension to emulate a WebXR device and its controllers, similar to the extension that Jaume creates a while back for WebVR: https://github.com/spite/WebVR-Extension

Some of the features we would like it to have:

Emulate WebXR so you can tests your application on your computer, no matter which VR capabilities or devices it has
Ability to move the camera and the controllers around, using keyboard, gamepad, mouse or any other device, as a gamepad controller
Ability to record and replay headset and controllers pose and actions
Ability to emulate different type of devices, like switching from 2x6dof controllers to 1x3dof controller and so

Takahiro (@takahirox) has been working for a while on a prototype of that extension as a proof of concept (Some of his tweets related to the extension): https://twitter.com/superhoge/status/1154514435450400768

https://twitter.com/superhoge/status/1143389540725379072

https://twitter.com/superhoge/status/1140575219721154561

https://twitter.com/superhoge/status/1140404713021685763

So we would like to get feedback on this idea and if people find this interesting we could move this development to the IW repos and keep discussing its features and roadmap in public.

avaer commented 4 years ago

Does this need to be an extension, or could it be a permissionless polyfill?

fernandojsg commented 4 years ago

@modulesio I believe it could be just a polyfill so you can inject as you want in your webapps, although installing an extension could be the most useful way to get it running in most users. I believe similar approach, having both options available is what they do in https://spector.babylonjs.com

avaer commented 4 years ago

The main thing we've learned doing the WebXR/WebVR device emulation over the last couple years is that there are a lot of cases where the web app wants to control the emulation. For example, the app has its own idea of saving poses and wants to replay them in 2d by virtually controlling a device. Or, remapping keyboard bindings that make most sense for the app's 2d interface.

The easiest path for that was a modifiable polyfill the app can include.

The extension can equally make sense, but there are use cases that I think it wouldn't cover alone.

fernandojsg commented 4 years ago

Yes, that could be one use case, where you need your app to know about the emulator itself and you adapt the app to integrate with the emulator.

The other use case that we have in mind is when apps know nothing about the emulator itself, so you can go to test one three.js or aframe example and you just launch the emulator so you can navigate them as it you had a headset and controllers, or you can use your real device to record a demo on a third party experience and send the recording to your coworker to test a bug when replaying that session, without need to modify the third party experience at all

takahirox commented 4 years ago

Let me add a new video https://twitter.com/superhoge/status/1159633171618549761

NellWaliczek commented 4 years ago

I'm definately excited about this extension and am, personally, comfortable with it moving into the immersive-web github org. The official person to check with is @TrevorFSmith though.

BTW, are y'all aware of the https://github.com/immersive-web/webxr-test-api work underway?

TrevorFSmith commented 4 years ago

This is clearly useful for almost everyone developing WebXR and I'm comfortable bringing it into an IW repo in a similar manner to the webxr-input-profiles work.

Since it has started somewhere else other than this org I believe that we'd need for the new repo to be created with one of the W3C-accepted licenses, perhaps this one, and then Mozilla and others could contribute under that coverage.

@dontcallmedom What do we need to do on the licensing/IP front to make this happen?

dontcallmedom commented 4 years ago

if I'm reading this correctly, the proposal here is not so much for a spec as much as code - if so, there is no particular worry in terms of IPR from a W3C adoption perspective - the license itself could be whatever feels most appropriate to the community.

(if there is a spec part to this, and that spec part has some chance to head to standardization in W3C at some point, then we would want to gather licensing commitments from the contributors - but that can be arranged once the said content has been migrated in any case)

TrevorFSmith commented 4 years ago

Ok, it sounds like we can create a repo and then use Issues there to work out use cases and how to structure the code for use in polyfills, WebExtensions, or some combination of the two.

How about "xr-emulator" as a name?

NellWaliczek commented 4 years ago

What's the latest on this? Is it moving into this org or staying where it is?

TrevorFSmith commented 4 years ago

@fernandojsg @takahirox Are you still up for creating the initial version of this in a Immersive Web CG repo? If so, I'll create it and give you write privileges.

TrevorFSmith commented 4 years ago

It looks like Mozilla is moving ahead in their own repo. @fernandojsg @takahirox

Should I close this Issue? I'm still happy to create a repo for it in the IW GitHub org if that's still what you'd like.

immersive-web / proposals

WebXR emulator extension #53