Access to 6dof positions without an immersive-ar session

[shanumante-sc]

In the current specification, in order to get 6dof pose (device position + orientation) from WebXR we must use immersive-ar session type and local/unbounded reference space.

However, using an immersive-ar session always opens a full-screen camera view which is not the desired behavior for our use case. Even with https://immersive-web.github.io/dom-overlays/ , the overlay is translucent and the camera is still visible full-screen underneath.

We would like to keep the original web-page visible and avoid the full-screen camera view, while still receiving 6dof poses which can be passed to our rendering engine.

With raw camera access (once the specification is implemented), we can use some hacks like clearing the framebuffer with constant white and use the overlays API, but this still leads to performance issues. At the same time, we need to build our UI to allow for smooth transition between immersive and non-immersive modes (since we don't always need 6dof poses for the entire lifetime of the webpage)

Taking the above into consideration, an ideal 6dof API for us:

1. Allows access to 6dof position with an inline session.
2. Allows access to users' webcam stream while the xr session is active.
3. Allows us to use our own rendering engine to augment the users' camera feed and display it back on the screen along with other HTML content without having to work around overlays API or such restrictions

Alternatively, an API like DeviceMotionEvent but for receiving 6dof data would also be a good alternative.

[grorg]

I support this feature request. I also don't know if it belongs as part of the DeviceOrientation spec.

[tangobravo]

I agree it would be great to expose native tracking outside of the immersive-ar session. I've written a long list of reasons why (immersive-web/webxr-ar-module#77) and proposed a couple of potential routes to do so (immersive-web/webxr-ar-module#78, immersive-web/webxr-ar-module#79).

DeviceOrientation / DeviceMotionEvent for me isn't the right spec - the data those specs expose only requires the accelerometer and gyro to be running, but for any sort of 6-DoF tracking the camera is needed too (just using the accelerometer alone leads to too much drift). Enabling the camera at the native device level has significant battery / CPU usage implications and will likely interact with getUserMedia in unexpected ways if it is just baked into DeviceMotion / DeviceOrientation.

[FilipePires98]

This would be useful for any immersive session (immersive-ar, immersive-vr).

[tangobravo]

This would be useful for any immersive session (immersive-ar, immersive-vr).

The immersive sessions already provide this capability. This issue is about providing access to 6-DoF tracking without requiring an immersive session.