Igalia / wolvic

A fast and secure browser for standalone virtual-reality and augmented-reality headsets.
https://wolvic.org
Mozilla Public License 2.0
813 stars 105 forks source link

XR Fragment support for native 3D file (glb e.g.) URL viewer #1435

Closed coderofsalvation closed 5 months ago

coderofsalvation commented 5 months ago

One of the frustrating things when it comes to webbrowsers, is that they limit their native filesupport to textual (html/txt) and max image/video (png/jpg/gif/mp4 e.g.). This limits the original vision of a 'hypermedia' browser.. Delegating anything else to javascript turned out to be not a great solution (privacy/security wise).

However, these days, one of the lowhanging fruits for an XR browser is to be able to natively view a 3D model file. I'd like to brainstorm a robust way of making Wolvic support native viewing of 3D .glb-URLs. Wolvic's home-scene is already displaying 3D objects. It should not be very hard to support URI's which use the opensource XR Fragments spec:

image

Solution:

Alternatives: one could argue for an browser extension-based approach, but that would defeat the soul of XR Fragments: surfing 3D files should be a safe and privacy-friendly 'right' for XR browser users.

Examples: you can fire up https://xrfragment.org/example/aframe/sandbox in your Wolvic browser and see how easy just some href and src can bring 3D models to life, and turn them into a web of VR/AR websites.

Disclaimer: I am a contributor of XR Fragments, but I would love to brainstorm on how to lower the barrier for VR designers to publish to a spatial web (by making Wolvic support XR Fragment URI's).

svillar commented 5 months ago

Thanks @coderofsalvation for the suggestions. I'd say that this should be really a proposal for web engines, either gecko or chromium (or both). Wolvic as a browser does not have anything to do with what the browser engine can handle (except for WebXR were we "inject" the data from the XR devices into the web engine)

coderofsalvation commented 5 months ago

Perhaps you're right, native file-handling does belong in the engine. The catch22 here is that I've noticed that chromium/gecko seem to be focused on hypertext-only, and defer anything non-text (3D) stuff to javascript api's. In short, the webengines are looking at the web through lens of (hyper)text (text embedding media). Personally, I think it's time XR devices should look at the web through the lens of (hyper)media (3D objects which can embed HTML e.g.). So basically the other way around:

webengine [3D rendering engine] -> 3D file -> media  (html  e.g.) -> scripting engine (js e.g)

instead of

webengine [2D rendering engine] -> HTML doc -> media (images) -> scripting engine (js e.g.) -> webgl -> load 3D file

It might be a crazy thought, but imho this would make more sense in various ways: security-wise, privacy-wise, survival-of-the-web-in-times-of-apps/game-engines. I know I'm trying to re-invent the web here, but honestly XR is kind of disrupting all things 2D anyways.

I guess I'm hitting an old 'what is a hypermedia browser'-debate here, or perhaps a fresh 'what should an XR webbrowser be like'-discussion. Back to XR Fragments: the IETF route (promoting the RFC there, in order to suggest to W3C) also seems like a risky investment of time. Not sure which is the best way to go about this. Perhaps I should first try coming up with a portable chromium-patch and a demo. Any thanks for the reply, any ideas are welcome.

svillar commented 5 months ago

Well IMHO that would be a complete revamp of the HTML5 model which I am not sure could happen in the short medium term even if you manage to convince all the involved actors.

Note that in the case of Wolvic we're already doing something similar to what you propose. Wolvic is really an app that uses a fairly simple and small 3D scenegraph library to render a 3D world in which we embed a traditional 2D browser which is capable of doing WebXR.

coderofsalvation commented 5 months ago

I totally agree, revamping the HTML5 is off the table. It's a bit of chicken/egg issue. You are basically describing Wolvic as a hypermedia browser which uses chromium/gecko to render HTML5 media (which phones home to the app via WebXR).

Some weeks ago, I've published a basic Godot Template to create 'XR hypermedia browsers' (with support for XR fragment URI's included), so technically I can publish an XR browser 'app' too (perhaps embed a HTML rendering engine too at some point):

https://xrfragment.org/#%F0%9F%A7%B0%20GODOT

Anyways, thanks for the brainstorm. Let me know in case you want to experiment beyond chromium/gecko's hypertext-lens, and allow native viewing of 3D model URLs too (usecases: set home-environment to https://wolvic.org/home.glb#pos=scene2, surf to https://svillar.homepage.net/testenvironment.glb#pos=attic e.g.) I'd be happy to chime in.

svillar commented 5 months ago

Anyways, thanks for the brainstorm. Let me know in case you want to experiment beyond chromium/gecko's hypertext-lens, and allow native viewing of 3D model URLs too (usecases: set home-environment to https://wolvic.org/home.glb, surf to https://svillar.homepage.net/testenvironment.glb e.g.) I'd be happy to chime in.

That sounds like exciting. Right now we have way too many things over the table, but we should seriously consider that for the future. Thanks for the comments!