Web Surfaces

[sirkitree]

https://github.com/Neos-Metaverse/NeosPublic/issues/60

[Xlinka]

https://github.com/CefNet/CefNet maybe the use of this cross platform implementation might work. For a chromium embedded browser.

[hazre]

Probably should use the available WebView on the OS.

[Xlinka]

Probably should use the available WebView on the OS.

does Linux Support it plus to add onto this issue

only the simulating user on the component should be running the sandboxed browser and be able to interact with it. so this makes it just a texture for other users where they can see the website that is currently being shown. (maybe a bool to allow other users to interact)

[hazre]

Yes every OS has it's own WebView.

On Win 10/11 it's WebView2 (Chromium) On most Linux distro's it's WebKitGTK On MacOS it's just WebKit.

There is libraries like this can automatically find the suitable WebView depending on OS: https://github.com/tauri-apps/wry

About that other issue, yeah that could work depending how you're going to turn it into a texture and send it over.

Alternatively you can just sync the WebViews together with just a simple WebRTC data channel. You just send the updates. Like on clicks, on scroll, on page update and so on. And you can hook into media API if something like video is playing on the screen and sync that automatically too.

[Xlinka]

i keep coming back and looking at this issue as if its a challenge but the problem would be interaction and security so requisites

• user allocation to run the browser
• only that user can interact with it
• allow the user to unlock it if they wanted to via a bool tick box to allow other users to use it (similar to the stream camera on allow anyone to interact)
• figure out how to pass the pointer of laser over to the web browser
• do we make generic ui like the base video player does ?

[hazre]

I've thought of about this more and the easiest way to do this is by streaming. The thing is syncing two or more webviews is too close to impossible, with weird artifacts, connection differences and etc. I think we should just do webrtc streaming and add a extra touch coordinates on top of it. So basically like a remote desktop and it would basically work like Neo's built-in desktop duplication. Only difference being that it's remote.

• I think a webview can be spawned by the client but everyone else will see a webrtc stream
• Optionally other users can interact with page. My guess is that we can use something like puppeteer to translate the clicks into actual action.
• Since it's webrtc it would be p2p which is good because no one needs to really host anything. (most internet speeds can handle 8 or so connections with something like a 1080p stream and if that's not enough, people can use their own SFU, that can act as a relay to put the strain on the server instead of the client)

I guess this needs to something closer to audio streams than let's a video player technically speaking. But idk how webrtc and neos streams would work. Maybe the plugin adds a webrtc client and that can pipe into Neo's stream (like audio stream) to display it in world space.