Define behavior when tab loses focus while presenting

[brianchirls]

What should happen when you switch to another tab or application while presenting in VR to an external display?

The spec says this:

In the event that the content process terminates unexpectedly, the browser will not exit VR mode. The VR compositor will destroy the content layer while continuing to present the trusted UI elements of the browser.

...which seems to indicate a desire not to abuse the user with a jarring experience. I think it's the right call.

It also says this:

The HMD pose and other VR inputs are only updated for the focused WebVR page. This can be implemented in the same manner as keyboard and mouse input.

The former seems to imply that there will be some pose-tracked 3D browser UI elements, so maybe the solution is to display that in place of the formerly active tab. So would you just resume presenting the web page as soon as it regains focus? What if you're switching between two tabs that are both VR enabled?

Any ideas?

[Sneagan]

This is a great conversation to have, but I wonder if it has a bearing on the spec. I fully anticipate there to be varying implementations of VR browser UI and window management, but they may be no more dictated by spec than the current window/tab management models that are implemented by browsers today. Is this use case categorically different from that?

[dmarcos]

What makes sense to me is that the active tab is the one presenting content on the HMD

[Sneagan]

I agree. I guess I interpreted the question to be asking how the transition between tabs works. That is, how do you:

1. Select a different tab
2. Animate into the new tab

[brianchirls]

I think it is relevant to the spec, because the spec already addresses what happens to a tab that does not have focus. So I think it needs to do that more completely. And I think it's different from traditional tabs because the cost of getting it wrong is much higher - abrupt loss of tracking can literally make one sick. And specs do dictate some of that behavior, like how the document.hidden property is set.

If you'll bare with me, I can think of a few scenarios where a tab might lose focus without exiting presentation:

1. You take off the HMD and you switch tabs manually. The headset is off, so there is no one to be disturbed by abruptly exiting VR. No big deal.
2. You have the HMD on while your friend comes along and switches tabs. It would be nice to recover safely from this scenario, but to be honest they could just as easily have turned off the computer. Your friend is a jerk, and I think that's outside the scope.
3. Another tab or application steals focus with some kind of pop-up dialog. This is the one I'm worried about because it's well outside the user's control and is often unpredictable. And it happens to me all the time - for example, if I have a reminder set in Google Calendar, that tab will show an alert that brings it into focus, hiding whatever I'm working on...

Therefore, I might propose that losing focus in the main 2d display does not interrupt VR presentation and that the security restriction about web page focus be amended so that pages that are not focused but are not presenting will still lose access to VR inputs, but once the page starts presenting, it retains access until it exits presentation.

Now, this raises the question of what happens if two different web pages try to present VR at once. My first thought is that this should be impossible. If a page tries to present on the same device while another page is already doing so, either that request should fail or the previous page should be forced to exit. I'm leaning towards the latter, since we already know that a presentation request can only be made by user input.

Okay, please shoot holes in my proposal, and then patch them.

[kearwood]

While the UX that might interrupt a VR site and the way it is animated in may be platform (Oculus Home or SteamVR) and/or Browser specific, I agree with the way @brianchirls describes the interaction with WebVR Sites.

I imagine that although only one WebVR site may enter VR at a time and can only lose VR focus by exiting presentation or by user interaction (i.e. following a link), a third state may occur. In this third state, the WebVR site will get orientation and positional tracking updates, but would not receive keyboard and button presses. The VR site would effectively be focused as normal but only receive VRDisplay VRPose tracking updates as required to continue rendering the site as the user moves their head. This third state would be possible, for example, when a non-modal dialog appears as a HUD that leaves the VR site visible behind it. Input would be blocked to avoid errant interactions; however, gaze based cursors models used in the content that do not require button presses to interact may inadvertently be triggered.

Browsers may avoid triggering such interactions in content while non-modal (hud) type dialogues are visible through clever designs such as attaching such hud dialogues to a motion tracked controller so the user can re-position them freely.

Such non-modal (hud) dialogues should not be used for security sensitive user input, such as password entry, as the VRPose updates could be used to infer the user's interactions with things such as an on-screen keyboard. In these cases, the browser could opt to present an opaque interface and suspend all input to the WebVR site until the user is no longer interacting with the interrupting UX.

WebVR content largely does not need to be aware of this behavior by the browser and platform. These are just examples of how a platform and browser may avoid issues and protect the user without affecting the WebVR API.

[cvan]

I'd encourage folks with a CV1 to try this flow:

1. From the Chromium builds, load a Web VR page and enter VR mode.
2. From the Oculus app, launch an app (e.g., Lucky's Tale).
3. Notice a dialogue screen appears asking if you want to continue even though chrome.exe is presenting.

I'm ignorant about what the SDK provides - can we do something similar?

[cvan]

Moved to w3c/webvr#28