sessiongranted API proposal

[dmarcos]

Initial API proposal for UA initiated immersive sessions. This enables in-vr navigation.

[ngokevin]

I think we've seen in the past that relying on a one-time browser events makes link traversal tricky, because the app has to register the handler before the browers fires it.

We can think of a more implicit mechanism such being able to enter XR with a flag that will work automatically once upon link traversal.

Or the browser detecting the registration of an event (e.g., registering a callback) that can automatically fire once attached.

[PlumCantaloupe]

Just wanted to comment on this proposal as very much appreciated. Traversing between pages, and staying in VR, has been my greatest headache in trying to create some educational content in WebVR; and the cost for popping out when it fails is high (loss of immersion is very jarring and feedback received has been extremely harsh when this happens). I don't have the resources to build my own architecture for linking between "worlds" and not leaving the page as it doesn't seem worth the benefit of using the web - may as well just go the native route. I definitely agree that having pages as "different scenes (i.e. Unity)" make developer friction much lower.

If there is a way to make this simple to develop with (event or the flag suggested above sounds nice) I feel it will go a long way towards developer buy-in beyond model viewers and demos.

[AlbertoElias]

Thanks for this Diego!

I wanted to bring my comment from the issue:

"I also remember from previous conversations the idea of a meta tag and/or HTTP Response Header. Browser vendors will be able to address this better, but could those optional add-ons improve performance? It does take a while to load and parse JavaScript, even more with WebXR where we probably have other libraries like Three.JS/Babylon.js/Polyfills."

I also really @Utopiah's comment on the issue: https://github.com/immersive-web/navigation/issues/1#issuecomment-477778097 There could be an "Enter VR mode and never leave it" on the browser level, and that would be another use case for the sessionGranted event.

The navigation explainer has a good overview of the missing sections in this PR.

Threats: https://github.com/immersive-web/webxr/blob/master/designdocs/navigation.md#potential-threats Same vs Cross origin: https://github.com/immersive-web/webxr/blob/master/designdocs/navigation.md#same-origin-vs-cross-origin-navigation

I think we can copy it over or link to the explainer

Would be interesting to hear thoughts on other people from the issue. And it's also valuable to hear actual developers and how they feel that this is essential and assuming that WebXR would not ship without it.

[dmarcos]

@AlbertoElias Thanks for the feedback.

Threats: https://github.com/immersive-web/webxr/blob/master/designdocs/navigation.md#potential-threats

I mentioned spoofing in the security considerations section. Happy to develop if needed. Maybe spam, undesirable, disorienting or nauseating content are existing problems of the Web and WebXR at large and not specific to navigation? Any additional problems particular to immersive navigation? Maybe we should address those risks in the WebXR spec in general?

I also remember from previous conversations the idea of a meta tag and/or HTTP Response Header

That's an option but not ideal from dev ergonomy point of view. It's error prone and often confusing based on our experience with the plethora of meta tags on mobile browsers. Would navigator.xr.requestSession succeed without user gesture if those headers or meta tags are present? Libraries will have to call requestSession at load and in some situations might fail and others succeed.

Browser vendors will be able to address this better, but could those optional add-ons improve performance?

Not sure if I understood perf implications. If it's about missing the event, UA could fire the sessiongranted event on attachment if needed. It would be interesting to see how other APIs handle events timing.

[AdaRoseCannon]

@AlbertoElias moving that into this repo would be ideal would you be able to open a PR to put them in here?

@dmarcos I can see UAs providing setting for whether this is a feature that is enabled by default. Security conscious users may want to decide whether pages can enter XR automatically and disable this feature via a setting in the user agent or via a 3rd party extension.

In addition I suspect website owners will want similar control for whether sites can continue in immersive mode if they have come from a 3rd party. For example an immersive bank may want users to sign in via a 2D interface but once they are in immersive mode navigate between internal pages maintaining the immersion. This kind of feature is something which would be ideally set via a HTTP header specifying referrer domains which allow for continual immersion.

[dmarcos]

@AdaRoseCannon Thanks for the input. It makes sense to give control mechanisms to UA, users and content creators.

Good thing is that sessiongranted is low friction and agnostic. It doesn't tie or prescribe a specific browser UX allowing experimentation and innovation in the space. Some browsers might decide to offer a traditional windowed workflow that fallback to 2D, others a more immersive experience. Content creators and library maintainers have to consider just two flows to cover all UAs: User and UA initiated immersive sessions via user gesture and sessiongranted event respectively. I'll add a note on that regard to the PR.

[AlbertoElias]

@AdaRoseCannon What's the best way for me to add it? Should I create a new markdown file?

@dmarcos I understand the developer experience isn't as good, but I don't mean adding the header or meta tag instead of the event. Just as a way to let the UA know in advance "hey, I'm going to be requesting immersive mode" or "never apply immersive mode when coming from a different origin" like Ada mentioned.

[AdaRoseCannon]

@AlbertoElias Yep make a PR for the new mardown file, assign me and i'll merge it :)

[dmarcos]

I added sections for cross-domain and security considerations in the PR. Happy to expand if necessary. I think It’s probably better for conversation to keep everything self contained in a single doc and reference external sources when appropriate.

[PlumCantaloupe]

A question related to this API proposal 😄

Could this same API/methodology be used to stay in fullscreen mode while traversing immersive links in Desktop mode? Also, allowing the same fullscreen functionality in mobile mode (i.e. remove that visually blocking URL bar? Sometimes I feel we forget how important it is to still support low-friction on these platforms within this "VR/XR transitional" period.

Or will we have to look towards purpose-built browsers like Supermedium or a Mozilla Reality browser deployed on smartphones and desktops? Maybe exokit?

[blairmacintyre]

@PlumCantaloupe

Could this same API/methodology be used to stay in fullscreen mode while traversing immersive links in Desktop mode? Also, allowing the same fullscreen functionality in mobile mode (i.e. remove that visually blocking URL bar? Sometimes I feel we forget how important it is to still support low-friction on these platforms within this "VR/XR transitional" period.

I don't see why not. What would you imagine the issue being that would prevent it?

Or will we have to look towards purpose-built browsers like Supermedium or a Mozilla Reality browser deployed on smartphones and desktops? Maybe exokit?

Most of the browsers share a bunch of common core (I think Supermedium is based on Firefox, right? Firefox Reality uses GeckoView?) so there's no reason that the base browser couldn't support similar things.

And exokit isn't a browser (it's a tool for prototyping/building native apps using web-tech, which is great for what it is).

[PlumCantaloupe]

Thanks for the reply @blairmacintyre. I suppose I haven't seen too many good examples of full-screen web applications and in the case of iOS a trend away from removing the URL by removing minimal-ui (though you can now do standalone PWAs I suppose). Was uncertain if there is some security-based ideology preventing the propagation of fullscreen control across web pages else why isn't it standard now.

[dmarcos]

@blairmacintyre Thanks for the thorough review. I pushed a bunch of changes addressing your comments.

[blairmacintyre]

Thanks for the reply @blairmacintyre. I suppose I haven't seen too many good examples of full-screen web applications and in the case of iOS a trend away from removing the URL by removing minimal-ui (though you can now do standalone PWAs I suppose). Was uncertain if there is some security-based ideology preventing the propagation of fullscreen control across web pages else why isn't it standard now.

There may be security concerns. Which is why this discussion is happening, I think. Popping out of immersive 3D in XR is more dramatic than exiting full screen, I think. Also, the extra UI required to deal with full-screen -> full-screen navigation probably doesn't justify it, given the lower incidence of full-screen apps.

[dmarcos]

Thanks for the reply @blairmacintyre. I suppose I haven't seen too many good examples of full-screen web applications and in the case of iOS a trend away from removing the URL by removing minimal-ui (though you can now do standalone PWAs I suppose). Was uncertain if there is some security-based ideology preventing the propagation of fullscreen control across web pages else why isn't it standard now.

There may be security concerns. Which is why this discussion is happening, I think. Popping out of There may be security concerns. Which is why this discussion is happening, I think. Popping out of immersive 3D in XR is more dramatic than exiting full screen, I think. Also, the extra UI required to deal with full-screen -> full-screen navigation probably doesn't justify it, given the lower incidence of full-screen apps.

Maybe interesting to consider browsers fullscreen mode triggered with F11 (@Utopiah mentioned in a different thread). URL disappears and fullscreen mode is preserved on navigation.

[cabanier]

Maybe interesting to consider browsers fullscreen mode triggered with F11 (@Utopiah mentioned in a different thread). URL disappears and fullscreen mode is preserved on navigation.

If you tell the browser to go full screen, you will still see the url bar. At least on my mac with Chrome, Firefox and Safari

[blairmacintyre]

@cabanier I think the F11 he's referring to is on Windows (I just tried F11 on Firefox and Edge and it does indeed go fullscreen). However, @dmarcos I don't think that this has anything to do with the Web Fullscreen API. It is simply the Windows 10 fullscreen-without-a-titlebar and happens to (for some reason) hide the URL bar. Seems like a platform choice, not something to do with the web APIs.

The Web Fullscreen API (ie. see demo at https://davidwalsh.name/demo/fullscreen.php) does not (I believe) stay in fullscreen when you navigate.

[dmarcos]

@cabanier I think the F11 he's referring to is on Windows (I just tried F11 on Firefox and Edge and it does indeed go fullscreen). However, @dmarcos I don't think that this has anything to do with the Web Fullscreen API. It is simply the Windows 10 fullscreen-without-a-titlebar and happens to (for some reason) hide the URL bar. Seems like a platform choice, not something to do with the web APIs.

The Web Fullscreen API (ie. see demo at https://davidwalsh.name/demo/fullscreen.php) does not (I believe) stay in fullscreen when you navigate.

Yes, It's not the same as the fullscreen API. I mentioned because it presents similar problematic to immersive navigation. There's no browser UI on screen and mode is preserved on navigation. Pages could potentially spoof the browser or the system UI. There's trusted ways to get back the UA interface and those differ across browsers. On Firefox, the user hovers the top of the screen to reveal the URL bar and Chrome shows a button to exit fullscreen mode all together.

[cabanier]

Yes, It's not the same as the fullscreen API. I mentioned because it presents similar problematic to immersive navigation. There's no browser UI on screen and mode is preserved on navigation. Pages could potentially spoof the browser or the system UI. There's trusted ways to get back the UA interface and those differ across browsers. On Firefox, the user hovers the top of the screen to peak the URL bar and Chrome shows a button to exit fullscreen mode all together.

That is odd. Are you on Windows? This is not how it shows on my mac.

[blairmacintyre]

Yes, @dmarcos is on Windows. Yes, Windows handles it differently than MacOS (that was the point of my message above).

The point of this discussion is to illustrate that their are different approaches to seemingly common ways of handling navigation in different instances on different operating systems. As Diego says, on Windows the browsers aren't insisting that the URL be shown all the time in a URL bar.

As with this and other aspects of the Web, there are things that are defined in the spec as required, and other things that are left up to the UA. This illustrates the point he (and others, including me) have been making that we should be open to doing similarly in XR, and not demanding a specific set of narrow behaviors from all browsers.

[cabanier]

Yes, @dmarcos is on Windows. Yes, Windows handles it differently than MacOS (that was the point of my message above).

It's surprising, but yes, if this is already a "blessed" mode, there's no reason why WebXR should be excluded. It would be good to hear if this mode was deemed a mistake that is now too late to remove.

It would be interesting to hear from a browser UX designer why this is not deemed an insecure workflow.

[arpu]

on linux chrome and firefox does not show the url bar in fullscreen mode F11

[dmarcos]

@AdaRoseCannon Sorry for the question. Still not familiarized with the protocol. How do you want us to move forward? Should we merge this and then submit more PRs open issues... etc?

[AdaRoseCannon]

If you're all happy i'll go ahead and merge it. Feel free to assign PRs to me for merging. :)

[AlbertoElias]

Thanks! What are the next steps? What´s the process to get this in browsers which is what us, content developers, are interested?

[AdaRoseCannon]

@AlbertoElias this is in step 3 of the transition process. Including documents to make the case for this feature would be very valuable.

Documents about security considerations, user journeys, getting feedback from developers is all very useful.

It may also be useful to contact @TrevorFSmith to ask about including it in a CG call to get more eyes from the Immersive Web Community.

[blairmacintyre]

@AdaRoseCannon I think a lot of that stuff has been dealt with (including, for example, in the explainer already in the webxr repo and the privacy and security repo). This is a small, well formed idea, unlike some of the other larger things being discussed in proposal repos (e.g., "computer vision" or even "anchors"). This has been significant discussion of these topics; perhaps more needs to be written, I'm not sure. What specifically folks want to see before moving this to the standard would be good.

I'd propose the next step is to get more of the group to look at it and open issues with questions, PR's with change suggestions. Can you add a small agenda item to todays meeting where you can say to folks "there's a draft README in the navigation repo, please go look and comment?". Then, we could take up a discussion in the CG call next week?

[TrevorFSmith]

I'm happy to add an agenda item to the next CG call (April 23rd) but it seems like we're making progress in this repo. One thing that needs to happen next is for the browser implementers to weigh in on whether the proposed API is missing important aspects (e.g. remaining security and privacy issues) and how they think about it fitting (or not fitting) into the current WebXR standards schedule. Hey @AdaRoseCannon, maybe you could reach out to the browser teams and get a sense of where they stand?