search

immersive-web / webxr

Repository for the WebXR Device API Specification.
https://immersive-web.github.io/webxr/
Other
2.99k stars 386 forks source link

Indicate "preferred" immersive mode #1360

Closed cabanier closed 3 weeks ago

cabanier commented 9 months ago

On most HMDs, the user can choose either a VR environment or passthrough. OpenXR offers an API to convey a preferred immersive mode; basically it returns if the user is in AR or VR. An application could use this to continue in this mode.

Maybe we can add an API to WebXR to do the same?

/agenda Indicate "preferred" immersive mode

AdaRoseCannon commented 9 months ago

This sounds really useful for when we get to navigation so a user can have a consistent experience.

toji commented 9 months ago

This is interesting, and I can see that it would be useful. I do have a few questions:

Would the preferred mode be the hardware preference or user preference? For example, it's easy to see that hardware like the Vision Pro would always prefer for users to stay in AR (putting aside technical issues preventing that right now), whereas a Quest 2 would probably prefer that users stay in VR, and a Quest 3 is likely not going to "prefer" one mode or another.

Rik mentioned that it would return if the user is already in AR or VR and return that, but thinking about that in the context of existing hardware that basically translates into "what's your home screen background?" and I don't see that being particularly meaningful in terms of how the content behaves? Maybe I'm wrong.

The navigation usage that Ada brings up is an interesting and separate issue that touches on permissions. Would we want to let the page choose to transition the user from a VR session to an AR session after navigation? That feels jarring, but I also haven't tried anything like that. If we are letting transitions like that happen then this feels like a good input to the navigation, but if not then this doesn't really contribute to the navigation story.

cabanier commented 9 months ago

This sounds really useful for when we get to navigation so a user can have a consistent experience.

I was thinking more about continuing in AR if your home environment is in passthrough but yes, we can also apply it to immersive navigation

cabanier commented 9 months ago

Would the preferred mode be the hardware preference or user preference?

At least for Quest, it would be a user preference. If they prefer the home environment in passthrough, maybe they want their WebXR experiences to be the same.

Rik mentioned that it would return if the user is already in AR or VR and return that, but thinking about that in the context of existing hardware that basically translates into "what's your home screen background?" and I don't see that being particularly meaningful in terms of how the content behaves? Maybe I'm wrong.

I'm unsure as well. I suspect the idea is that when possible, we want transitions to be more seamless. (For instance, we also did work to remove black screen between passthough home and passthrough WebXR)

AdaRoseCannon commented 8 months ago

/facetoface moving to the face to face since we didn't get time to cover in the call

dennyronngren commented 8 months ago

Hi, I just came across this thread. I work for Varjo (who manufactures mixed reality HMDs with video passthrough) and also participate in the OpenXR working group. I think that one of the ways to indicate current runtime / user preference is the order of the blend modes that the runtime supports in the xrEnumerateEnvironmentBlendModes call, I do know that there are other ways than setting the environment blend mode, but think that for devices that does support that way of enabling MR, the xrEnumerateEnvironmantBlendMode order could be used for application to understand which one to pick (based on current state of the device). Is there a place to follow the updates on this from the face to face?

cabanier commented 8 months ago

Hi, I just came across this thread. I work for Varjo (who manufactures mixed reality HMDs with video passthrough) and also participate in the OpenXR working group. I think that one of the ways to indicate current runtime / user preference is the order of the blend modes that the runtime supports in the xrEnumerateEnvironmentBlendModes call

We already have support for that using the isSessionSupported API. This proposal is about what the preferred mode is.

apowers313 commented 8 months ago

When we were working on the W3C WebAuthn spec I think we got feedback that features like this could be used in aggregate to "fingerprint" specific users and track them. Not sure if a binary "prefer vr / ar" could really do much harm, but something to consider.