Does not work with environment map

[donalffons]

Hi and thanks a ton for this great feature in Google maps / this example!

I was trying to use an environment map (with metalness / roughness - values) on my model and noticed that this doesn't seem to work with WebGLOverlayView. The reflections of the environment map on the object are static and are not updating correctly. I'm not sure if the problem is in my setup or in WebGLOverlayViewor in ThreeJS.

The video shows the result with Google Maps and WebGLOverlayView on top and the result with "normal" ThreeJS on the bottom.

https://user-images.githubusercontent.com/33040347/163732934-43f46c00-e179-4f71-b8cb-06707f3764f5.mp4

Here are my change to your code:

    // load the model    
    loader = new GLTFLoader();
    const source = "pin.gltf";
    loader.load(
      source,
      gltf => {
        gltf.scene.scale.set(25, 25, 25);
        gltf.scene.rotation.x = 180 * Math.PI / 180; // rotations are in radians

        scene.add(gltf.scene);

+        // Load Environment Map
+        new EXRLoader().load("/furry_clouds_2k.exr", texture => {
+          const pmremGenerator = new THREE.PMREMGenerator(renderer);
+          const exrCubeRenderTarget = pmremGenerator.fromEquirectangular(texture);

+          // Apply Environment Map
+          gltf.scene.traverse(o => {
+            if (o.material) {
+              o.material.envMap = exrCubeRenderTarget.texture;
+              o.material.envMapIntensity = 1;
+              o.material.metalness = 1;
+              o.material.roughness = 0.1;
+              o.material.needsUpdate = true;
+            }
+          });
+        });
      }
    );

It's a pretty simple code change and I'm clueless why it's behaving like this. Please help!

PS: I also noticed that your GLTF has all inverted faces. It's not visible in your demo, but if you start changing the material settings, it becomes noticeable. Do you want a PR :slightly_smiling_face: ?

[donalffons]

Alright, I figured out that this is because only the projectionMatrix of the camera is updated - but not the position / rotation / (scale?!). If I update the camera position and rotation with values from projectionMatrix.clone().invet().decompose(position, rotation, scale), I can see the environment map works "correctly". But now my model is sliding across the planet, which isn't ideal...

What would be the correct way to get the camera's transformation matrix in global coordinates?