search

immersive-web / webxr

Repository for the WebXR Device API Specification.
https://immersive-web.github.io/webxr/
Other
3k stars 386 forks source link

Confusion around actual and internal visiblity #1396

Open cabanier opened 1 month ago

cabanier commented 1 month ago

An external developer asked us to send better visibility events. They want to pause video as soon as the user takes off the headset but currently, the visibility event doesn't happen until the headset sits idle for a short while.

This is because we tie the WebXR session visibility to the visibility state of the internal OpenXR session and not to if the headset is on the user's head.. The WebXR and OpenXR specs both state that visibility is tied to if the user can see the scene, but that is not the case for Quest. In addition, Quest can still report poses even if the display is turned off but the session is still active.

The WebXR spec states: A state of hidden indicates that imagery rendered by the XRSession cannot be seen by the user. requestAnimationFrame() callbacks will not be processed until the visibility state changes. Input is not processed by the XRSession.

so if I tie visibility to the mounted state of the headset, that would violate the spec (because we keep firing raf calls). I also worry that changing this will also break some sites.

Alternatively, I can add a boolean on the session that states if the headset is currently worn and update the normative text that says that visible or visible-blurred frames may not be shown to the user (which is Quest's current behavior). Thoughts, @AdaRoseCannon or @toji

toji commented 1 month ago

My gut says that the headset not being mounted feels like an appropriate signal to switch to the hidden visibility state. XRSession RaFs and input could be suppressed if needed to ensure spec compat. I'd be interesting in hearing under what conditions you feel like that would break content.

Separate from that, I think any language which explicitly states "shown to the user" might be worth reconsidering the phrasing for. Perhaps something like "displayed by the device" is more appropriate, since it doesn't imply that the user is actually seeing the content. (Mostly I don't want to give anyone ideas to get too clever and start firing "hidden" state changes because their fancy eye-tracking device could tell that the user has their eyes closed...)

cabanier commented 1 month ago

My gut says that the headset not being mounted feels like an appropriate signal to switch to the hidden visibility state. XRSession RaFs and input could be suppressed if needed to ensure spec compat. I'd be interesting in hearing under what conditions you feel like that would break content.

If the browser stops submitting frames when it's not mounted, the openxr machinery won't transition the openxr session to visible if the user puts the headset back. I guess I could work around that on the browser side but all the assumptions in the blink code about not submitting frames when invisible would have to be deleted or updated.

If the site makes an early return in its raf call (if !visible-> return) and doesn't submit frames, the site will never transition to visible since that is still tied to the openxr state. I don't believe I can fix that.

Separate from that, I think any language which explicitly states "shown to the user" might be worth reconsidering the phrasing for. Perhaps something like "displayed by the device" is more appropriate, since it doesn't imply that the user is actually seeing the content.

Yes, that would be a good change.

cabanier commented 3 weeks ago

/agenda do we need an event for headset donning/doffing?