search

immersive-web / dom-overlays

A feature incubation repo for layering DOM content on/in WebXR content. Feature lead: Piotr Bialecki
https://immersive-web.github.io/dom-overlays/
Other
70 stars 9 forks source link

[Comment] Best practices for overlays #1

Closed nbutko closed 4 years ago

nbutko commented 5 years ago

When designing our WebXR UIs, our design team likes to consider whether UI elements are diagetic (characters / objects in the scene are aware of their presence) or non-diagetic (visible only to the user) when we consider whether they should be 2d dom elements or 3d elements embedded in the scene.

From this point of view, it might be useful to think of dom overlays that are attached to the document body body as "outside" of the immersive scene, and shown through a platform appropriate HUD type experience, and it's up to the platform to figure out what this means.

klausw commented 5 years ago

I think that's a good way of looking at it. The proposed DOM overlay isn't intended to appear integrated into the world view or 3D scene, it's only showing UI elements or informational text to the user. In a multiuser scenario, each user would have their own private copy of this.

A core aspect of the DOM overlay is that there is no assumption about how it's rendered relative to the 3D view, and this is what gives the user agent that flexibility. It would be possible to have the user agent report a current world-space location of the DOM overlay back to the application on a frame-by-frame basis, but that risks having applications make assumptions that would break cross-form-factor compatibility.

Going beyond this, I think that in-world DOM content would still be useful, and the separate layers proposal has the potential of addressing that use case. My hope is that future implementations with full support for arbitrary layers would be able to provide the simpler DOM overlay API also as a special case fairly easily. Even if arbitrary layer support is available, it may be useful for applications to work with a DOM overlay since that lets the user agent take care of form factor differences.

Apart from the specific API shape, there are general challenges to exploring DOM rendering paths and input handling, and I think that exploring these by trying out implementation experiments would be helpful in informing a variety of different approaches.

klausw commented 4 years ago

Closing, I think this matches how the API is currently described.