Does WebXR support basic single screen 3D displays such as a 3D DLP Projector?

[kintz09]

I've been doing a lot of reading about the current state of WebXR and I'm looking for help answering a compatibility question. We have a 3D web application built with BabylonJS. The application currently supports consuming the same content in standard 2d web, HMDs, and the zSpace display.

We are working towards migrating our customers from our Unity Standalone app to the newer web application. To accomplish this, we need to add support for single screen fixed displays such as a 3D DLP Projector, 3D monitor, or 3D TV.

Does the WebXR Device API offer support for these types of devices currently? If so, could someone provide me with a link to more information or some key terms to use when searching for this information?

If not, are there plans to add this type of support? I've seen a few mentions of CAVES in this project which lead me to believe this may be possible, especially considering a 3D projector is basically just a one-walled CAVE.

[cabanier]

Do you send 2 different pictures to get a stereo effect to these devices? If so, WebXR supports this.

[klausw]

I'm not aware of specific current implementations that specifically support non-head-mounted 3D displays, but I may not be up to date on this. The WebXR API is intended to be able to support a variety of immersive technology, but as far as I know the user agent implementations are currently focused on VR/AR headsets along with smartphone AR.

For implementations that are based on OpenXR, these may work on new form factors without specifically added browser support as long as the underlying devices support OpenXR.

A CAVE system combines stereo projection with head pose tracking to ensure that the scene is shown consistently for a specific viewer's position. The WebXR API represents this as a collection of views that each have a virtual camera pose (position and orientation) that should in general match one of the user's eyes. For your use case of a 3D projector, would you be intending to use this with head tracking, or just for a stationary arbitrary assumed viewer position?

There's some flexibility for the API, for example adding an auxiliary view for a third party observer intended to be shown on a flat monitor, but in general there's an assumption of having an overall tracked user pose along with an input device. I think it should be possible to use the WebXR API for scenarios that don't match this, and this could work well for custom apps targeting those form factors specifically, but it would likely not work well for arbitrary existing applications.

[Maksims]

The way most 3D Projectors work today, is projecting polarizing image onto the screen, by alternating left/right images with actively synchronized glasses. Although they usually take 2D video signal input, which is simply split-screen. Where projector then takes that 2D signal and and stretches it, alternating one half then the other.

So, technically, no even WebXR APIs are required, two viewports (each eye - viewport) rendering - all is needed. Most WebGL engines have functionality to render multiple cameras with custom viewports. This can be further optimized using OVR_multiview2 extension to reduce drawcalls.

[cabanier]

It's cleaner to go through the WebXR API because then you can draw the scene to the projector while still drawing the page to the regular monitor. @kintz09 you should be able to use WebXR to drive these types of devices.

[kintz09]

Do you send 2 different pictures to get a stereo effect to these devices? If so, WebXR supports this.

Yes! We render the right and left eye separately and then the GPU takes over to convert it to whatever format the device expects. Historically we've utilized Quad-buffered OpenGL rendering (first in Virtools, then Unity) to achieve our stereo effect. The most common format used by our devices is Frame Sequential or Frame Packing, but it would also work with devices that support Top-Bottom, Side-by-Side, Vertical/Horizontal interlaced, etc.

We are already utilizing the WebXR API to for zSpace and HMD devices, so I'm hopeful supporting these types of devices won't be overly complicated. I think we mostly need to get a better understanding of how to pass things off to the GPU.

[kintz09]

I'm not aware of specific current implementations that specifically support non-head-mounted 3D displays, but I may not be up to date on this. The WebXR API is intended to be able to support a variety of immersive technology, but as far as I know the user agent implementations are currently focused on VR/AR headsets along with smartphone AR.

For implementations that are based on OpenXR, these may work on new form factors without specifically added browser support as long as the underlying devices support OpenXR.

A CAVE system combines stereo projection with head pose tracking to ensure that the scene is shown consistently for a specific viewer's position. The WebXR API represents this as a collection of views that each have a virtual camera pose (position and orientation) that should in general match one of the user's eyes. For your use case of a 3D projector, would you be intending to use this with head tracking, or just for a stationary arbitrary assumed viewer position?

There's some flexibility for the API, for example adding an auxiliary view for a third party observer intended to be shown on a flat monitor, but in general there's an assumption of having an overall tracked user pose along with an input device. I think it should be possible to use the WebXR API for scenarios that don't match this, and this could work well for custom apps targeting those form factors specifically, but it would likely not work well for arbitrary existing applications.

That's pretty incredible the WebXR API has the functionality to power a CAVE!

We do have a version of our application that works in a CAVE with head-tracking and other inputs like a wand, but those are more custom. At this time, I'm really just looking to support the stationary arbitrary assumed viewer position. The vast majority of these customers use a DLP Link projector with active glasses and no head tracking. Our most common use case is a teacher/professor presenting to a classroom of students and everyone is wearing 3D glasses.

[kintz09]

It's cleaner to go through the WebXR API because then you can draw the scene to the projector while still drawing the page to the regular monitor. @kintz09 you should be able to use WebXR to drive these types of devices.

That goes beyond our requirements at this time, but we know this will be a highly requested feature, so it's fantastic to know this will be possible.

[kintz09]

@cabanier, @klausw, @Maksims Thank you so much for your wonderful and very timely responses!

I think it's safe to say the WebXR API can definitely drive our application for a fixed viewport single screen stereo device.

I'm comfortable saying that my team knows how to handle the cameras, matrices, viewports, etc. to render the left and right eye into a texture properly. Could anyone help guide me on how to finish connecting the dots with the GPU?

Does the WebXR Device API go as far as handling everything with the GPU? Where does the WebXR API end and something else take over?

[klausw]

The WebXR Device API exposes JavaScript interfaces using constructs such as poses, viewports, and an opaque framebuffer layer used as a WebGL render target. The user agent (browser) implementation is responsible for translating that to an appropriate lower-level API to ensure that the drawn pixels end up on the output device, and that the poses and viewports match the low-level API's spatial tracking state. See the explainer for a sample render loop.

The tricky bit for your issue is that adding WebXR support for a custom device either requires a customized web browser with support for that added, or using a common lower-level API such as OpenXR if the browser supports that. For example, Edge and Chrome on Windows support OpenXR and can in theory support any OpenXR-compatible immersive device, though in practice it's entirely possible that the current code may be making assumptions such as "there will be two views corresponding to left/right eye" that may be incorrect for a system such as a CAVE.

Does the WebXR Device API go as far as handling everything with the GPU? Where does the WebXR API end and something else take over?

I think you bring up a good point, I'm not sure if there's good documentation explaining these distinctions. The explainer is rather high-level, while the specification quite quickly dives into specification terminology and algorithms that aren't the most user friendly. There's a section on primary and secondary views which may help.