Simplification: Enable single-canvas inline, defer desktop mirroring

[toji]

I'd like to propose that we seriously evaluate allowing a single-canvas method of rendering inline content, deferring the mirroring feature on desktop, (and as a result also deferring the XRPresentationContext and XRRenderState.outputContext attribute) till after the core spec is complete.

Reason: I think I've personally heard more developers express confusion about how XRPresentationContext works and why it's required for inline sessions than just about any other feature in the API. Devs seem to understand it slightly better when talking about mirroring, but for inline there's a lot of confusion as to why the output doesn't simply go to the original WebGL canvas by default. I frequently end up explaining it in terms of more advanced use cases with galleries of sessions on a single page, but the fact of the matter is that most developers are not going to be pursuing that type of use case, and thus it just sounds like additional work for no benefit to them.

Additionally, the requirement to use an XRPresentationContext has been very problematic for developing an efficient polyfill of inline features. It forces an awkward series of copies to try and get the image from the WebGL context to the fake 'xrpresent' context that can absolutely tank performance. (It's not as much of an issue for immersive sessions, since the polyfill doesn't need to support mirroring on mobile and has to do a copy anyway to get the lens distortion correct.) If we could drop the two-canvas requirement it would make polyfilling the API core much more performant and practical.

For mirroring, we find that pretty much every native API has a built-in mirroring capability. For desktop APIs there's usually a way to open a mirroring window for development, debugging, or streaming. For mobile devices there's often a way to stream to an external device like Chromecasts, which our API won't be able to provide anyway, and as such the mirroring feature is semi-redundant. (And removing it does not prevent spectator rendering or demo-driver use cases.)

Finally, some native implementations (such as Windows Mixed Reality and OpenXR) don't provide a mirror texture, so those platforms would either have to do a second composite in browser in order to show a proper mirror or just show nothing, in which case the mirroring isn't a dependable feature anyway (and we should make sure that we communicate when it's available an when it's not.)

Proposal: We could add a method of creating a WebGL layer that specifically indicates that the WebGL default backbuffer should be used rather than an XR-specific framebuffer. For example, let's assume that it's an option for the XRWebGLLayer constructor:

// This is a terrible option name and I will die of shame if we actually use it.
let xrLayer = new XRWebGLLayer(session, gl, { useWebGLDefaultBackbuffer: true });

This layer now has several unique properties: All of the framebuffer creation options (framebufferScaleFactor, alpha, depth, etc.) are ignored, because the target framebuffer already exists. The layer's framebuffer attribute will be null and the framebufferWidth and framebufferHeight just reflect the gl.drawingBufferWidth and gl.drawingBufferHeight. This means that you can bind to it and use it like you would a normal WebGL layer in your drawing loop, but you'll just be rendering straight into the WebGL canvas instead. Also, because of that, when we start supporting multiple layers we'd restrict it so that layers of this type cannot be mixed with any other layers since we wouldn't actually be doing any compositing here. It's just a "passthrough" layer of sorts. (And no. That can't be the option name. Because AR.) Finally, for obvious reasons, you couldn't create a layer with this option for immersive sessions.

The restrictions above sound annoying, but the overall reduction in complexity of both the spec and implementation should be significant. The WebGL behavior is basically free, because it's largely just letting WebGL do what it already does and simply leveraging the layer as means for unifying the rendering of simple, first-gen WebXR content.

Forward Compatibility: I want to preserve the capability to add mirroring/advanced inline features back in shortly after the core API is locked down because there ARE cases where it's useful/necessary. Developers who want to show multi-layer content inline would have to use an XRPresentationContext, as would developers that want to use a single context with multiple sessions. But at least then we would be saying "You have to do this because you need these features" Rather than "You're forced to jump through this hoop because someone else might need to do something fancier some day."

We need to ensure that it can properly be feature detected and could be added back in without interfering with the core. Given that canvas.getContext() is specced to return null for unknown context names, that gives devs a really clean feature detect point to say "hey, is more advanced inline/mirroring available?" Additionally there will be several detectable attributes (such as XRRenderState.outputContext) that developers would be able to check for at that point.

Conclusion: This is a large change and probably feels kind of hacky upon initial reading, but I strongly encourage us to look for a way to achieve the above goals, even if my specific proposal isn't taken up. Again, this is probably the single most complained-about aspect of the API in my experience, and I think that we have an opportunity to make the lives of ourselves and developers simpler here.

[cwilso]

Need to decide in May, although implementation (in spec) may take longer.

[TrevorFSmith]

I heartily agree that the presentation context should be opt-in for experiences that want to mirror and that it can be a post-initial-spec feature. It's the number one source of confusion when I go over the API with developers.

[jsantell]

Broadly, I think this solves the polyfill-overhead in inline sessions. Yay!!