search

immersive-web / navigation

Repository for the discussion and research in to navigating from page to page whilst staying in immersive mode. Feature leads: Rik Cabanier and Brandon Jones
Other
70 stars 9 forks source link

Any update ? #9

Open felixmariotto opened 4 years ago

felixmariotto commented 4 years ago

Now that webXR is stable and widely implemented, what are the possibilities for content creators who would like to let their user navigate between pages without leaving immersion ?

Any news regarding your specs ? Any hack in the meantime ?

AdaRoseCannon commented 4 years ago

/agenda lets discuss this on the next CG call to see what current interest looks like :)

felixmariotto commented 4 years ago

Where can I be updated and participate to discussions on the matter ? On this repo ?

AdaRoseCannon commented 4 years ago

Yes, on this repo.

On Wed, 22 Apr 2020, 13:31 felixmariotto, notifications@github.com wrote:

Where can I be updated and participate to discussions on the matter ? On this repo ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/immersive-web/navigation/issues/9#issuecomment-617749955, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABAHSMU3VL472VBGSYA242TRN3PQRANCNFSM4MN77YKA .

PlumCantaloupe commented 4 years ago

Thank you for bringing this up.

This is an issue near and dear to my heart as having multiple worlds connected as web pages is a much simpler way to build multiple environments than trying to load/unload somehow with js. We build up so many immersive features in our HMDs to improve presence and then we throw it all away without a way to traverse worlds in VR here.

Would like to contribute somehow to getting this in. My research is in using VR (WebXR) for learning, but I have often bothered The A-frame and Oculus Browser teams often enough to hack this in when it breaks on updates (they are doing a spectacular job btw) that I can at least engage in discussion here as a developer/user.

felixmariotto commented 4 years ago

We build up so many immersive features in our HMDs to improve presence and then we throw it all away without a way to traverse worlds in VR here.

Exactly. Without immersive navigation, websites that need external links will necessarily be user-hostile.

Let's jot down a list of a few popular websites and determine if a VR feature / equivalent to these websites with immersive navigation would be possible and desirable :

Per the above listing, we can state that linking to external websites will be an important feature of the VR web, as important as in the normal web, if not more. Now the question is, is the current state of navigation allowing content creators to build those desirable websites ? I am designer by training, so while not able to help a lot technologically speaking, I might at least give the point of view of a content creator in a consistent way. Those are the reasons why in my opinion the current navigation won't allow for good web design :

Because of all these reasons, the current navigation between VR experiences is long, complex, and error-prone. Therefore, the users will avoid it as much as possible.

I hope this clarification will be useful in convincing (if necessary) that, in this time of wide VR adoption since last Christmas, a solution should be found for immersive navigation.

PlumCantaloupe commented 4 years ago

These are all great points. Ultimately, it is severely limited if we have to ask users to press a couple more buttons so I think your “Affordance” and “Accessibility” themes are the most problematic.

Being able to easily connect “worlds/pages” will likely open up some interesting immersive design possibilities unimagined currently.

One interesting note is that you mention control of transitions. I don’t believe that is easily possible as that would require a a non user action to delay navigation to new page before loading. It is a nice feature to think about though, and I wonder if that is something worth thinking about as part of spec. or best practise.

Currently, with the Oculus Quest things kind of load in asynchronously which can cause frame rate hiccups and ugly model/texture loading. Loading screens are so ubiquitous for all web apps so is it time to consider some sort of browser-based feature for knowing when things are loaded/not-loaded, especially as most VR/3D content is going to be high bandwidth.

dmarcos commented 4 years ago

FYI, The Oculus Browser team did an amazing job shipping an experimental implementation of this proposal. There was a small bug that slipped in @Artyom17 Do you know if it's been addressed? We can incorporate in A-Frame as soon as there's a functional implementation. Additional context on https://github.com/aframevr/aframe/issues/4445

qster123 commented 4 years ago

I liked JanusVR's solution to this. A point of entrance was specified by the creator and anyone could create a link to the website address, this was represented as a round portal into the creator's room.

Dante83 commented 4 years ago

Probably not an idea anyone else would care to implement. But when I thought of this, I always figured that web developers could provide interfaces that you could integrate to hook two worlds together. When you create your world, you define a set of vertices and colors that reflect the edges and the person wishing to hook into that world would be responsible for creating a set of vertices that aligned with that edge and had the same colors/textures. Then they could always do what they wanted after they satisfied that interface. This would give the impression of an 'open world' but what worlds were contained within that open world would be dependent upon which interfaces you wish to combine. Of course, such a world would naturally be limited by geometry. Individuals could probably also do this with portals if they just made them into a cube map and went out of their way to align the vertices to a standard 'sim size' between worlds. That said, with a bit of non-euclidean magic, you might be able to 'get away' with more then you might otherwise be allowed to. Perhaps there could be ways to make 'blurry edges' to grant wiggle room between region sizes?

mhenry07 commented 4 years ago

What should a baseline, recognizable link be on the immersive web? The equivalent of <a href="destination-url">description</a> with no css.

How should a baseline link look and interact? What are the minimum properties required? Do WebXR authors just add a LinkNode to a Scene and set a couple properties?

How should destination properties be encoded and decoded - in URL fragments (#)? There could be predefined named sub-destinations or possibly source specifiable position and orientation. Would the spec define how browsers would load the named sub-destinations and/or specified viewer transform, or would WebXR authors have to implement one or both of those themselves?

I would love dioramas with small gltf or cubemap previews of destination environments. Or perhaps a responsive preview that loads on demand. That's probably not going to be practical for baseline links. But maybe there can be considerations for handling common use cases.

felixmariotto commented 4 years ago

@dmarcos this is super interesting, thank you for sharing. @Artyom17 I read that :

the "Default" setting allows only certain well-known domains, so, most likely you need to switch to Enabled state to make it work for all domains.

Can you please mention some of these well-known domains, so we can test the feature ? Are you planning on allowing it for all domains by default eventually ?

AdaRoseCannon commented 4 years ago

The topic around representing one scene in another is related to what this repo is trying to solve but is not the critical problem to be solved through web standards as it probably should be solved at higher level such as an agreed convention which 3D libraries choose to adopt together.

The tricky issue is navigating from a scene on one web site and entering another whilst maintaining immersion. In a way that can be trusted, is secure and a good experience for users.

Assets like GLTF or equirect/cubemap previews could go a long way in assisting browsers help make the process seamless but is definitely not what the blocker has been when this issue has been brought up in the past.

AdaRoseCannon commented 4 years ago

It's been 2 months since this was last discussed at the Face to Face in February, here are the minutes: https://www.w3.org/2020/02/06-immersive-web-minutes.html#item05

PlumCantaloupe commented 4 years ago

It seems security is the main concern here?

If so, when moving cross-domain could there not be some browser implemented modal that pops up in VR (not removing you from your existing virtual space) and asks if you consent?

AdaRoseCannon commented 4 years ago

That is the issue, known as "trusted ui" how do you know the pop up box is one from the OS or browser that you can trust and not an accurate simulacrum created by the VR experience, or worse something malicious created by a 3rd party.

PlumCantaloupe commented 4 years ago

Ah, I see thank you. Hmm, would reserving an area temporarily, that a developer can’t write graphics into, in VR make sense? Maybe all navigation (or any other user awareness triggers) triggers a 360 “white bar” consent graphic that cannot be overwritten?

It would so nice to figure this out as the methodology could be extended to hiding the URL bar for true full-screen web experiences on mobile/desktop.

felixmariotto commented 4 years ago

would reserving an area temporarily, that a developer can’t write graphics into, in VR make sense?

We can boil it down to this, the problem is only to have the browser display a user interface that the user can be 100% sure it is the browser's, but ideally can be hidden. As you wrote, it could be extended to full-screen navigation on mobile and desktop.

In my opinion the issue would be addressed by enforcing these rules (though I have no idea how they could be enforced) :

This way, the only possible action to hide the interface would be to press the reserved input. Therefore, if a malicious website tries to simulate a change of page with a false UI showing up, the illusion would fall apart when the user press the input to hide it, since it would show up the real UI with the real URL.

The issue I can foresee with this, is if the user is presented a false browser UI, and choose not to hide it. Ideally this UI would prompt the user a lot for hiding it, by various means depending on the support, like staying in the middle of the screen, or the field of view.

dmarcos commented 4 years ago

FYI, there's a security considerations section in the current proposal I recommend reading. We can expand or modify if necessary

HeadClot commented 4 years ago

Just thought I would chime in on this. I know that Exokit and JanusXR has done something very similar to what navigation is trying to do. Maybe Immersive web group could save some work and time by getting the devs from Exokit onboard for this feature?

Anyway - A few questions for the immersive web group.

  1. Will there be artist focused tooling apart from pure HTML and CSS? I might be able to give you some ideas for this. If you are open to artist focused tooling.
  2. How do you plan on keeping large worlds running at 90 FPS or higher? I know that Exokit is managing this by using chunking the world up and aggressive use of LOD's.
  3. Will this work with WebGPU?
  4. Will this work with Electron and NWjs?

I really want to see this become a standard for the web.

avaer commented 4 years ago

Speaking for Exokit, I support the proposal and we just implemented it in XRPackage.

However I think tooling is way out of scope for the navigation spec, and perhaps even W3C. The fact that navigation specification enables meta-engines to be built on top is an independent issue and the first step is probably to get sessions transitioning in existing browsers and frameworks like THREE.js and A-Frame.

mhenry07 commented 4 years ago

Related to a trusted popup/modal/overlay, would replacing representation of inputs (hands, controllers, etc) be part of that? It might lessen immersion but might communicate that this is part of the trusted UI. Would all user input (including hand location, etc) need to be blocked from application/content while trusted UI is open? Head position/orientation should probably not be blocked, but should all other device inputs be temporarily intercepted by trusted UI?

dmarcos commented 4 years ago

This proposal describes the minimal mechanism for sites and user agents to implement immersive navigation. The spec can indeed contain a list of security considerations and general solutions. The UX particulars are probably better left to the browser vendors to innovate. It’s still early days and will will faster converge to good solutions based on real world usage.

dmarcos commented 4 years ago

@Artyom17 Thanks! Looking forward to this. Are there any plans to open up to any domain? If not is it possible to add glitch.com for the community to experiment?

Artyom17 commented 4 years ago

@Artyom17 Thanks! Looking forward this. Are there any plans to open up to any domain? If not is it possible to add glitch.com for the community to experiment

The issue is that WebXR workgroup doesn't have agreement on this, so, technically speaking it is still an experimental feature with certain security issues. Therefore, we do not have plans to open it widely by default (however, you can always enable navigation for any domain by changing an option in chrome://flags).

The issue with adding glitch.com is that it doesn't have one responsible owner. This is like to add whole github.com to the list.... See above about security issue with current navigation implementation.

dmarcos commented 4 years ago

@Artyom17 I see, thanks. Would be acceptable enabling by default on whitelisted domains and also adding a flag for the user to enable manually for any domain? That’s an approach used by Chrome with experimental features and would allow the community to also experiment

avaer commented 4 years ago

I'm not sure what the process is but this feels like it could function as an Origin Trial.

dmarcos commented 4 years ago

To clarify I meant an entry in chrome://flags to enable vr navigation disabled by default for all non-white listed domains. Having an origin trials kind of program for Oculus Browser would be also indeed very useful. Thanks again for all your work

haywirez commented 4 years ago

How does this work right now in the context of single page JS apps and window.location.history.pushState? Does the user remain in VR context if a “page reload” doesn’t occur?

Artyom17 commented 4 years ago

@Artyom17 I see, thanks. Would be acceptable enabling by default on whitelisted domains and also adding a flag for the user to enable manually for any domain? That’s an approach used by Chrome with experimental features and would allow the community to also experiment

If I understand correctly - it is already exactly like this. Navigation is enabled by default for the whitelisted domains, and users can enable navigation for any domain via chrome://flags "WebXR Navigation permissions" (set it to "Allowed").

dmarcos commented 4 years ago

@Artyom17 Thanks. That's awesome to know. I'll test and merge functionality in A-Frame tomorrow.

asajeffrey commented 4 years ago

I've been holding off on implementing immersive navigation due to the lack of a spec. Is the Oculus implementation based off of the sessiongranted examples in the README?

I'm a bit nervous about an implementation that uses a curated list of sites, and is enabled by default, it seems like it's likely to result in walled gardens. Could we make the default that same-site navigation (in the sense of eTLD+1) is enabled rather than a curated list?

Artyom17 commented 4 years ago

Totally agreed that the hardcoded whitelist is not a great solution, and we are considering to implement something like google's origin trials (no ETA, though). As to enabling the same site navigation by default: unfortunately, it still has the same security issues of lacking trusted UI and malicious sites can try to mimic sensitive UI in attempt of stealing user's login/password... IMO

asajeffrey commented 4 years ago

Are there plans to implement the features of origin trials that protect against experimental features from becoming de facto standards?

https://github.com/GoogleChrome/OriginTrials/blob/gh-pages/explainer.md#a-sketch-of-a-solution says that origin trials are opt in for any site (not a curated list) and are monitored to keep usage below 0.5%.

jaydave1412 commented 4 years ago

I was also struggling with this issue so I solved that by wrapping the elements in a scene entity and using XHR requests to change those elements without reloading the scene and showing some general loading animation while the scene is loading also XHR request work with a-assets so you can load assets dynamically as well I am very new at this so I don't understand most of the things stated in above discussion but this is how I solved this issue

Edit: it only works for in site navigation example(https://blinklinksolutions.com/vero%20kitchens%20virtual%20tour/store.php)

dmarcos commented 4 years ago

FYI, we integrated the proposal on A-Frame. You can try today on the Oculus browser via master builds. An example you can try: https://glitch.com/edit/#!/roomy-silly-fork?path=goaland.html%3A17%3A65

Thanks everybody that has been participating. This is exciting

coderofsalvation commented 3 years ago

Hi all

Superexcited about this. I'm afraid, with current user-numbers, security will stay somewhat of a theoretical problem, not a practical one (we don't have userdata/feedback). Two practical things i can think of now, are:

  1. Consent in the form of a UA flag (chrome://settings, about:config e.g.) which unlocks hasslefree link-traversal for webxr (sessiongranted).

  2. Piggyback the solution found in browser extensions: manifest.json. Webxr developers can limit the untrusted effect by specifying it in their https://mywebxrapp.com/manifest.json:

{
  "short_name": "My app",
  ...
  "trusted_interfaces":{
    "version":1,
    "vrdisplay":{
      "matches":["https://otherwebxrapp.com"]         // adding 'https://*/*' would trigger consentscreen by UA
    },
   "fullscreen":[
      "matches":["https://otherwebxrapp.com/*"]`     // adding 'https://*/*' would trigger consentscreen by UA
    ]
  }
}

The manifest.json is already parsed by browsers anyways, and can also be parsed by webxr-developers, so there's a common ground for innovation. This way the developers can also choose what to go with (the UA solution or their own). Basically mid-to-side evolution instead of top-down browser-to-developer evolution.

I'm aware of the fact that I & we might be overlooking a lot of things here in retrospect. However, as an webxr user & developer, imho semantical linking of webxr apps is a huge essential. Such an essential, that it seems odd to wait for the evolution of cross-UA consentscreen-specs before the 'web'-part is put in WebXR. Cross-UA consentscreens are such a rabbithole, just look at its evolution for 2D websites. My few cents: I'd rather have 'sessiongranted' link-traversal available as a flag in the meantime, and act upon its data afterwards.

cabanier commented 3 years ago

@coderofsalvation please don't paste the same comment in different issues. You can put a reference to your other comment here.

coderofsalvation commented 3 years ago

Thx. Will do from now on (I wasn't aware of github providing links to a comment vs the whole thread ).