[Feature Request] Equivalent to JanusVr's websurfaces for embedding websites in iframes

vesper8 commented 5 years ago

I apologize in advance if this is the wrong place to ask.

I have been wanting to find a way to embed a website via an iframe in Aframe for some time. I know this is not possible. I've read how WebGL cannot render html elements such as Iframes.. and that the way to do it would be to overlay an iframe on top of the canvas and position it in such a way that it looks as though it's part of the scene.

However today I stumbled upon this template on JanusVR and I was pretty blown away https://vesta.janusvr.com/spyduck/neon-website-translator-template

It's a 3D WebVR scene that pulls in a fully interactive website inside an iframe

Inspecting the code you can see there is a div underneath the

<canvas/>
    <div
      class="engine_systems_render_css3d"
      style="overflow: hidden; width: 1056px; height: 317px; perspective: 206.561px;"
    >
      <div style="transform-style: preserve-3d; width: 1056px; height: 317px; transform: translateZ(206.561px) matrix3d(-1, 0, 0, 0, 0, -1, -0.0001, 0, 0, 0.0001, -1, 0, 0, 1.8, 0.00018, 1) translate(528px, 158.5px);">
        <div
          class="janusweb_websurface "
          style="width: 1920px; height: 1080px; position: absolute; transform: translate(-50%, -50%) matrix3d(0, 0, -0.00182292, 0, 0, -0.00180556, 0, 0, 1, 0, 0, 0, -3.37, 2, 4.5, 1);"
        >
          <iframe
            src="https://spyduck.net/index.html"
            allow="vr"
            style="width: 1920px; height: 1080px;"
          />
        </div>
      </div>
    </div>

The iframe loads a regular website, not some kind of html shader. And the website can be interacted with.

The amazing part is that as you move around in the scene, the iframe's parent div is transformed so that it always seems like the iframe is part of the scene. I am really impressed by all of this.

I believe the magic here is called "Websurfaces" which is a feature of JanusVR.

It would be incredible if Aframe could support something like this.. either through a plugin or in the core. What do you think? Is this feasible? Or is there a technical reason why it can't be done?

dmarcos commented 5 years ago

It would be cool. Unfortunately, it’s not something that A-Frame (or any other WebGL / WebVR framework) can do properly without additional Web standards like for example an iframe to texture API. There’s no way to render arbitrary HTML into WebGL besides some limited hacks.

Janus is using CSS 3D transforms to overlay an iframe in the same coordinates reference system as the WebGL scene. The iframe is always painted on top, oclussion with WebGL doesn’t work, it can’t be shaded / blended with the rest of the scene and it won’t work in VR mode (only magic window / 2D mode). Notice how the iframe is no longer rendered when clicking the VR button because only WebGL content can be presented in headset.

A 3rd party component could implement something similar to Janus Websurface but given the VR focus of A-Frame it does not fit as a core feature. We will have to wait until new standards come along.

vesper8 commented 5 years ago

Ah I see.. I was wondering how that worked when you entered VR mode.. but I guess it still looks super cool in Magic Window / 2D mode

It would be very nice if a plugin brought this functionality to Aframe. Too bad Janus's implementation isn't open source...

I hope someday this is possible.. embedding iframes into WebGL I mean.

Thanks for the answer!

aframevr / aframe

[Feature Request] Equivalent to JanusVr's websurfaces for embedding websites in iframes #3933