openwebdocs / project

project documentation, policies & meeting minutes
http://openwebdocs.org
Creative Commons Zero v1.0 Universal
359 stars 42 forks source link

OWD Project Proposal: WebXR Browser Compatibility #35

Closed JoelMarcey closed 2 years ago

JoelMarcey commented 3 years ago

WebXR Browser Compatibility

MDN has an essential feature where it provides browser compatibility data for its documented web technologies, including web APIs, JavaScript and HTML. This compatibility is usually represented via a table of browsers, and is essential reading for developers to ensure their apps will be usable across the widest variety of browser platforms.

Currently, the represented browsers in these tables are the common and widely used desktop and mobile browsers. And while this data is of course necessary because of the broad use of these browsers, there are other browser categories that should also be considered when it comes to compatibility.

Virtual and augmented reality (VR and AR, collectively “XR”) devices are becoming more common in households and the industry. These devices have web browsers and WebXR provides the functionality and APIs to the web to those devices. In fact, WebXR is becoming so mainstream that MDN is starting to officially document it.

Proposal

Given the increasing use of XR devices, XR should be added alongside desktop and phones in the compatibility tables on MDN. Mozilla Mixed Reality and Facebook’s Oculus Browser are examples of browsers for XR devices which could appear here. We expect browsers which work on desktops and phones, but support XR features, would continue to appear under their primary modality of desktop/phone.

Each browser vendor would provide the necessary data, in the JSON necessary format, to be added to the compatibility repo. And then that data would be exported into the MDN documentation.

The proposal is as follows:

  1. An extra column would be added to the compatibility tables called “XR Devices”, with an icon such as Slice 1, where native XR-based browsers would be listed.
  2. Facebook will work to provide Oculus Browser compatibility data in browser-compat-data. Facebook would like Oculus Browser to appear in the XR Devices column.

Note: Facebook would like feedback on what level of coverage OWD needs to see for (2).

Facebook has noticed some recent XR features are missing from browser-compat-data. Facebook can update missing XR-specific features as it publishes its compatibility data. This will make it easier for other vendors and community contributors to publish updated compatibility data for XR features in other browsers.

Elchi3 commented 3 years ago

We've met last Wednesday (June 9th, 2021) to assess this project. We think it's suitable for us to work on this topic and it seems reasonably scoped to take on within a quarter. The following priority assessment was made:

Priority assessment

This table checks this project against the OWD prioritization criteria.

Criteria Assessment Points
Effort Medium/Large. Comments:
For better estimation: Figure out how much data we need where
Maybe get a sample data set
Need to figure out what this means for BCD first, then how to render it
Also, how much is engine data vs browser specific data?
Workload can be shared given Facebook can help with the work and other immersive Web folks likely, too
-2
Dependencies Figuring out how this fits into BCD
Writing WebXR docs (can be a separate project afterwards or in parallel)
-1
Community enablement Enables the Immersive Web community 1
Momentum High. The core spec is stable, devices become more common in households and the industry 2
Enabling learners N/A 0
Enabling professionals Medium to high. Professionals need to know the compatibility of XR tech and to what extend features are available 1,5
Underrepresented topics / ethical web N/A 0
Operational necessities N/A 0
Addressing needs of the Web industry We think this addresses needs very much, but no survey or other data points have been presented to show a high demand 1
Elchi3 commented 3 years ago

I've started to get more familiar with WebXR and as a first step I took a look at the existing MDN WebXR reference pages. I made a pass to clean them, to show compat tables properly, and have a consistent structure. (no dictionaries, enums, callback pages anymore -- this was wrong according to the MDN reference guidelines and BCD data structures). This should be all in shape now.

MDN and the compat data mark almost everything related to WebXR as "experimental". Here I would appreciate some subject matter advice which parts of the API should really be considered "experimental"/"unstable" and which not.

@dominiccooney and I also chatted a bit about landing pages and we should clean them up some more and give a better overview of the core WebXR Device API and all the available extensions/modules.

I've also identified missing reference pages and I think a good next step is for me to create them. This is the list of features where we know (thanks to the compat data) that implementations exist but MDN docs haven't been written yet. Of course, if other WebXR browsers implement more features, please let me know and also add it to the compat data!

(done) ~XRAnchor~ (done) ~XRAnchor.anchorSpace~ (done) ~XRAnchor.delete~ (done) ~XRAnchorSet~ (done) ~XRCPUDepthInformation~ (done) ~XRCPUDepthInformation.data~ (done) ~XRCPUDepthInformation.getDepthInMeters~ (done) ~XRDepthInformation~ (done) ~XRDepthInformation.height~ (done) ~XRDepthInformation.normDepthBufferFromNormView~ (done) ~XRDepthInformation.rawValueToMeters~ (done) ~XRDepthInformation.width~ (done) ~XRFrame.createAnchor~ (done) ~XRFrame.getDepthInformation~ (done) ~XRFrame.getHitTestResults~ (done) ~XRFrame.getHitTestResultsForTransientInput~ (done) ~XRFrame.getLightEstimate~ (done) ~XRFrame.trackedAnchors~ (done) ~XRHitTestResult~ (done) ~XRHitTestResult.createAnchor~ (done) ~XRHitTestResult.getPose~ (done) ~XRHitTestSource~ (done) ~XRHitTestSource.cancel~ (done) ~XRInputSource.gamepad~ (done) ~XRLayer~ (done) ~XRLightEstimate~ (done) ~XRLightEstimate.primaryLightDirection~ (done) ~XRLightEstimate.primaryLightIntensity~ (done) ~XRLightEstimate.sphericalHarmonicsCoefficients~ (done) ~XRLightProbe~ (done) ~XRLightProbe.onreflectionchange~ (done) ~XRLightProbe.probeSpace~ (done) ~XRRay~ (done) ~XRRay.XRRay~ (done) ~XRRay.direction~ (done) ~XRRay.matrix~ (done) ~XRRay.origin~ (done) ~XRSession.depthDataFormat~ (done) ~XRSession.depthUsage~ (done) ~XRSession.requestHitTestSource~ (done) ~XRSession.requestHitTestSourceForTransientInput~ (done) ~XRSession.requestLightProbe~ (done) ~XRTransientInputHitTestResult~ (done) ~XRTransientInputHitTestResult.inputSource~ (done) ~XRTransientInputHitTestResult.results~ (done) ~XRTransientInputHitTestSource~ (done) ~XRTransientInputHitTestSource.cancel~ (done) ~XRView.isFirstPersonObserver~ (done) ~XRView.recommendedViewportScale~ (done) ~XRView.requestViewportScale~ (done) ~XRWebGLBinding~ (done) ~XRWebGLBinding.XRWebGLBinding~ (done) ~XRWebGLBinding.getDepthInformation~ (done) ~XRWebGLBinding.getReflectionCubeMap~ (done) ~XRWebGLDepthInformation~ (done) ~XRWebGLDepthInformation.texture~

Elchi3 commented 3 years ago

What's missing in the compat data (and also on MDN) are features implemented by Oculus.

(done) ~XRCompositionLayer~ (done) ~XRCompositionLayer.layout~ (done) ~XRCompositionLayer.blendTextureSourceAlpha~ (done) ~XRCompositionLayer.chromaticAberrationCorrection~ (done) ~XRCompositionLayer.mipLevels~ (done) ~XRCompositionLayer.needsRedraw~ (done) ~XRCompositionLayer.destroy~ (done) ~XRProjectionLayer~ (done) ~XRProjectionLayer.textureWidth~ (done) ~XRProjectionLayer.textureHeight~ (done) ~XRProjectionLayer.textureArrayLength~ (done) ~XRProjectionLayer.ignoreDepthValues~ (done) ~XRProjectionLayer.fixedFoveation~ (done) ~XRQuadLayer~ (done) ~XRQuadLayer.space~ (done) ~XRQuadLayer.transform~ (done) ~XRQuadLayer.width~ (done) ~XRQuadLayer.height~ (done) ~XRQuadLayer.onredraw~ (done) ~XRCylinderLayer~ (done) ~XRCylinderLayer.space~ (done) ~XRCylinderLayer.transform~ (done) ~XRCylinderLayer.radius~ (done) ~XRCylinderLayer.centralAngle~ (done) ~XRCylinderLayer.aspectRatio~ (done) ~XRCylinderLayer.onredraw~ (done) ~XREquirectLayer~ (done) ~XREquirectLayer.space~ (done) ~XREquirectLayer.transform~ (done) ~XREquirectLayer.radius~ (done) ~XREquirectLayer.centralHorizontalAngle~ (done) ~XREquirectLayer.upperVerticalAngle~ (done) ~XREquirectLayer.owerVerticalAngle~ (done) ~XREquirectLayer.onredraw~ (done) ~XRCubeLayer~ (done) ~XRCubeLayer.space~ (done) ~XRCubeLayer.orientation~ (done) ~XRCubeLayer.onredraw~ (done) ~XRSubImage~ (done) ~XRSubImage.viewport~ (done) ~XRWebGLSubImage~ (done) ~XRWebGLSubImage.colorTexture~ (done) ~XRWebGLSubImage.depthStencilTexture~ (done) ~XRWebGLSubImage.imageIndex~ (done) ~XRWebGLSubImage.textureWidth~ (done) ~XRWebGLSubImage.textureHeight~ (done) ~XRWebGLBinding.nativeProjectionScaleFactor~ (done) ~XRWebGLBinding.createProjectionLayer~ (done) ~XRWebGLBinding.createQuadLayer~ (done) ~XRWebGLBinding.createCylinderLayer~ (done) ~XRWebGLBinding.createEquirectLayer~ (done) ~XRWebGLBinding.createCubeLayer~ (done) ~XRWebGLBinding.getSubImage~ (done) ~XRWebGLBinding.getViewSubImage~ (done) ~XRMediaBinding~ (done) ~XRMediaBinding.XRMediaBinding~ (done) ~XRMediaBinding.createQuadLayer~ (done) ~XRMediaBinding.createCylinderLayer~ (done) ~XRMediaBinding.createEquirectLayer~ (done) ~XRLayerEvent~ (done) ~XRLayerEvent.XRLayerEvent~ (done) ~XRLayerEvent.layer~ (done) ~XRRenderState.layers~

(done) ~XRInputSource.hand~ (done) ~XRHand~ (done) ~XRJointSpace~ (done) ~XRJointSpace.jointName~ (done) ~XRFrame.getJointPose~ (done) ~XRFrame.fillJointRadii~ (done) ~XRFrame.fillPoses~ (done) ~XRJointPose~ (done) ~XRJointPose.radius~

(done) ~XRPose.angularVelocity~ (done) ~XRPose.linearVelocity~

Elchi3 commented 3 years ago

Compared the core spec's IDL with what we have on MDN. A few (new) features that are undocumented came up and it's unclear if anyone implements them (yet).

XRSession.frameRate XRSession.supportedFrameRates XRSession.updateTargetFrameRate XRSession.onframeratechange

(done) ~XRWebGLLayer.fixedFoveation~

dominiccooney commented 3 years ago

These are in various stages but all of these are experimental or shipped in some browser: Oculus Browser has the frame rate APIs behind an experimental flag; we think Chrome for Android implements the viewport scale, some browsers have the property but it's a no-op; Oculus Browser has shipped fixedFoveation but I think we are the only one. (Caveat: My knowledge of Samsung Internet's VR support and Hololens/Edge's VR support is very limited.)

Elchi3 commented 3 years ago

I've finished most of this work. Q3 comes to an end and over the course of this quarter, I have:

My mandate here still includes to help getting the Oculus browser compat data into BCD and getting the missing support statements in. @dominiccooney is driving this, though. The issue is https://github.com/mdn/browser-compat-data/issues/12303.

After that's done, I'll close this issue and work on something new. See https://github.com/openwebdocs/project/issues/55 for Q4 planning which also includes a new WebXR-related project: WebXR guides. If you are an OWD Steering Committee member, you can vote it up so it will be considered.

I would like to thank Ada Rose Cannon (@AdaRoseCannon, Samsung Internet), Dominic Cooney (@dominiccooney , Facebook), and Joe Medley (@jpmedley, Google) for helping with reviews and code examples and their support in the #webxr-docs OWD slack channel.

JoelMarcey commented 3 years ago

Thanks for the great work everyone! This is so awesome to see.

AdaRoseCannon commented 3 years ago

This is totally incredible work, thank you for all of it!! I am sure it is already being a massive benefit for WebXR Developers.

On Thu, 23 Sep 2021, 20:17 Florian Scholz, @.***> wrote:

I've finished most of this work. Q3 comes to an end and over the course of this quarter, I have:

  • documented features from 10 WebXR specifications
  • wrote 131 new MDN reference pages
  • submitted 78 pull requests to MDN for this work
  • submitted 4 pull requests to immversive-web specifications/explainers
  • converted the MDN WebXR docs from HTML to Markdown

My mandate here still includes to help getting the Oculus browser compat data into BCD and getting the missing support statements in. @dominiccooney https://github.com/dominiccooney is driving this, though. The issue is mdn/browser-compat-data#12303 https://github.com/mdn/browser-compat-data/issues/12303.

After that's done, I'll close this issue and work on something new. See

55 https://github.com/openwebdocs/project/issues/55 for Q4 planning

which also includes a new WebXR-related project: WebXR guides https://github.com/mdn/content/issues/7276. If you are an OWD Steering Committee member, you can vote it up so it will be considered.

I would like to thank Ada Rose Cannon @. https://github.com/AdaRoseCannon, Samsung Internet), Dominic Cooney ( @dominiccooney https://github.com/dominiccooney , Facebook), and Joe Medley @. https://github.com/jpmedley, Google) for helping with reviews and code examples and their support in the #webxr-docs OWD slack channel.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openwebdocs/project/issues/35#issuecomment-926086166, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABAHSMSQ4PDZUQVRF6KKBYLUDN4LXANCNFSM45BEPH6A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Elchi3 commented 2 years ago

Closing this but I'm encouraging the WebXR community to file new issues with OWD if there are new things we should add to MDN/BCD.

Adding Oculus to BCD/MDN is discussed in the following issues/PRs and is in @dominiccooney's hands.
https://github.com/mdn/browser-compat-data/pull/12642 https://github.com/mdn/browser-compat-data/issues/12303

An issue about more WebXR guide material is https://github.com/mdn/content/issues/7276