GLB frosted glass material only marginally affect primitives, doens't affect <a-text> or <a-image>

[nickverneck]

Description: I imported an object created in blender where it has different index of refraction and transmission to show as a frosted glass however, it only affect primitives like boxes and spheres but it doesn't affect itself, as if I put a copy of it behind, it doesn't affect text, and it doesn't affect images, so pretty much any element that is 2D doesn't get blur. I tried playing with the material and changing parameters but I still can't get refraction to work.

• A-Frame Version:1.5.0
• Platform / Device: chrome/quest2
• Reproducible Code Snippet or URL:

• 

`<a-box position="0 -0.3 -4"></a-box>
      <a-gltf-model id="postContent" src="#blenderModel" scale="1.5 1.5 1.5" position="0 0 -50" visible="false" pinch-click>

      </a-gltf-model>
<script>
        const modelEl = document.querySelector("a-gltf-model#postContent");

        modelEl.addEventListener("model-loaded", () => {
          console.log("event triggered");
          const model = modelEl.getObject3D("mesh");
          console.log("event model", model);
          model.traverse((child) => {
            console.log("event child", child);
            if (child.isMesh) {
              const material = child.material;

               //Modify material properties
               material.ior = 1.9; // Index of Refraction
               material.roughness = 0.3; // Roughness
               material.transmission=1;

               material.color.set("gray"); // Diffuse color (red)
                // Emissive color (green)

              material.needsUpdate = true;
            }
          });
        });
      </script>`

[mrxz]

The transmission pass of Three.js doesn't render transparent objects. By default both the <a-text> and <a-image> primitives are marked as transparent. For images you can set transparent="false". However, the built-in text rendering does effectively need to be transparent.

Note that it isn't recommended to use transmission when targeting stand-alone/mobile VR headsets as the transmission passes cause expensive operations that impact performance too much. For context there is ongoing work to improve the transmission pass in Three.js (see https://github.com/mrdoob/three.js/pull/28078), but it'll obviously take time before it lands in Three.js let alone A-Frame, assuming the changes get incorporated.

[nickverneck]

I see, Setting transparent=false end up getting the a-text and a-image material blur in general, but viewing from behind an object also with transmission they now don't get rendered at all.

I tried to write my own shades and custom materials as well but I wasn't successful . However, I'm not sure if this is a limitation on three.js transmission as I've seen in reactXR doing this exact effect on their examples. https://codesandbox.io/p/sandbox/router-transitions-4j2q2?file=%2Fsrc%2FApp.js

I wonder if this could be ported to a-frame since they are using THREE.physicalMaterial under the hood. https://github.com/pmndrs/drei?tab=readme-ov-file#meshtransmissionmaterial

Thanks for explaining btw!

[mrxz]

Under the hood A-Frame is pretty much vanilla Three.js, so anything that can be done with Three.js can be done with A-Frame with some effort as well. Looking at the MeshTransmissionMaterial from drei you linked, it seems that it specifically supports "seeing" other transmissive and transparent objects through it, so that might just be what you're looking for.

I don't know if it properly works in WebXR, though. So that's something you'll have to find out if you attempt porting it.