WebGLRenderer replaces WebGPURenderer texture exceptions

[Jinxishihenian]

Description

When I load the gltf model with WebGLRenderer, everything works fine with the textures. But when I replace WebGLRenderer with WebGPURenderer, some of the textures in the model are abnormal.

With WebGPURenderer, the texture results are different even for each refresh

Reproduction steps

1.Use WebGLRenderer to load gltf correctly 2.Replace WebGLRenderer with WebGPURenderer 3.Texture anomalies

Code

import React from 'react';

import ReactDOM from 'react-dom/client';
import './index.css';
import * as THREE from 'three';

import WebGPURenderer from 'three/examples/jsm/renderers/webgpu/WebGPURenderer';

import { OrbitControls } from 'three/examples/jsm/controls/OrbitControls';
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader';
import App from './App.tsx';

ReactDOM.createRoot(document.getElementById('root')!).render(
    <React.StrictMode>
        <App />
    </React.StrictMode>
);
window.onload = async () => {
    // 创建场景
    const scene = new THREE.Scene();

    // 创建相机
    const camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000);
    camera.position.set(0, 1, 2);

    // 使用WebGPURenderer
    const renderer = new WebGPURenderer();
    // 使用THREE.WebGLRenderer
    // const renderer = new THREE.WebGLRenderer();

    renderer.setSize(window.innerWidth, window.innerHeight);
    document.body.appendChild(renderer.domElement);

    // 创建轨道控制器
    const controls = new OrbitControls(camera, renderer.domElement);
    controls.update();
    // 创建一个立方体
    const geometry = new THREE.BoxGeometry();
    const material = new THREE.MeshBasicMaterial({ color: 0x00ff00 });
    const cube = new THREE.Mesh(geometry, material);
    scene.add(cube);

    // 调整窗口大小
    window.addEventListener('resize', () => {
        camera.aspect = window.innerWidth / window.innerHeight;
        camera.updateProjectionMatrix();
        renderer.setSize(window.innerWidth, window.innerHeight);
    });
    const loader = new GLTFLoader();

    // 调整窗口大小
    window.addEventListener('resize', () => {
        camera.aspect = window.innerWidth / window.innerHeight;
        camera.updateProjectionMatrix();
        renderer.setSize(window.innerWidth, window.innerHeight);
    });

    scene.background = new THREE.Color(0x88ccee);
    // scene.fog = new THREE.Fog(0x88ccee, 0, 50);
    // 添加环境光.
    scene.add(new THREE.AmbientLight(0xffffff, 1));
    // 添加平行光.
    const directionalLight = new THREE.DirectionalLight(0xffffff, 1);
    directionalLight.position.set(200, 200, 200);
    directionalLight.castShadow = true;
    directionalLight.shadow.camera.near = 0.01;
    directionalLight.shadow.camera.far = 500;
    directionalLight.shadow.camera.right = 30;
    directionalLight.shadow.camera.left = -30;
    directionalLight.shadow.camera.top = 30;
    directionalLight.shadow.camera.bottom = -30;
    directionalLight.shadow.mapSize.width = 1024;
    directionalLight.shadow.mapSize.height = 1024;
    directionalLight.shadow.radius = 4;
    directionalLight.shadow.bias = -0.00006;
    // directionalLight.intensity = 200;

    scene.add(directionalLight);
    // 添加方向光助手
    const directionalLightHelper = new THREE.DirectionalLightHelper(directionalLight, 6);
    scene.add(directionalLightHelper);
    // 半球光.
    const fillLight = new THREE.HemisphereLight(0x8dc1de, 0x00668d, 1);
    fillLight.position.set(2, 1, 1);
    scene.add(fillLight);

    loader.load(
        // 模型路径
        '/assets/gltfs/ZhanTing625/SM_ZhanTing625.gltf',
        // 加载完成的回调函数
        function (gltf) {
            scene.add(gltf.scene);
        },
        // 加载过程中的回调函数
        function (xhr) {
            console.log((xhr.loaded / xhr.total) * 100 + '% 已加载');
        },
        // 加载出错的回调函数
        function (error) {
            console.error('加载模型出错', error);
        }
    );

    // 渲染循环
    function animate() {
        requestAnimationFrame(animate);
        controls.update();
        renderer.render(scene, camera);
    }

    animate();
};

Live example

• WebGLRenderer

• WebGPURenderer

Screenshots

No response

Version

0.165.0

Device

Desktop

Browser

Chrome

OS

Windows

[Mugen87]

Do you mind sharing the glTF asset that produces the issue?

[Jinxishihenian]

Sorry, the source file is a company asset and I can't provide it, but I have a similar gltf file as follows (same texture issue caused by WebGPURenderer): https://github.com/mtsee/vr-hall/tree/main/public/assets/room1

Also, I think it's necessary to inform about the anomalies I've found:

1. When I use WebGPURenderer, the browser console prompts (WebGPURenderer.js?v=6a0b6706:38 THREE. WebGPURenderer: WebGPU is not available, running under WebGL2 backend.）
2. Based on the gltf file of my company's source file, when I deleted the other models and only left the chair, the chair texture was rendered correctly.

[Mugen87]

The first warning is self-explaining, it just means you render with the WebGL backend.

We should get rid of the second one before investigating the issue more closely. When you switch to WebGPURenderer, please use setAnimationLoop(). This method can also be used with WebGLRenderer and is the recommended way to define the animation loop.

Do you still see the issue with this updated version of your app code:

import React from 'react';

import ReactDOM from 'react-dom/client';
import './index.css';
import * as THREE from 'three';

import WebGPURenderer from 'three/examples/jsm/renderers/webgpu/WebGPURenderer';

import { OrbitControls } from 'three/examples/jsm/controls/OrbitControls';
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader';
import App from './App.tsx';

ReactDOM.createRoot(document.getElementById('root')!).render(
    <React.StrictMode>
        <App />
    </React.StrictMode>
);
window.onload = async () => {
    // 创建场景
    const scene = new THREE.Scene();

    // 创建相机
    const camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000);
    camera.position.set(0, 1, 2);

    // 使用WebGPURenderer
    const renderer = new WebGPURenderer();
    // 使用THREE.WebGLRenderer
    // const renderer = new THREE.WebGLRenderer();

    renderer.setSize(window.innerWidth, window.innerHeight);
    renderer.setAnimationLoop(animate);
    document.body.appendChild(renderer.domElement);

    // 创建轨道控制器
    const controls = new OrbitControls(camera, renderer.domElement);
    controls.update();
    // 创建一个立方体
    const geometry = new THREE.BoxGeometry();
    const material = new THREE.MeshBasicMaterial({ color: 0x00ff00 });
    const cube = new THREE.Mesh(geometry, material);
    scene.add(cube);

    // 调整窗口大小
    window.addEventListener('resize', () => {
        camera.aspect = window.innerWidth / window.innerHeight;
        camera.updateProjectionMatrix();
        renderer.setSize(window.innerWidth, window.innerHeight);
    });
    const loader = new GLTFLoader();

    // 调整窗口大小
    window.addEventListener('resize', () => {
        camera.aspect = window.innerWidth / window.innerHeight;
        camera.updateProjectionMatrix();
        renderer.setSize(window.innerWidth, window.innerHeight);
    });

    scene.background = new THREE.Color(0x88ccee);
    // scene.fog = new THREE.Fog(0x88ccee, 0, 50);
    // 添加环境光.
    scene.add(new THREE.AmbientLight(0xffffff, 1));
    // 添加平行光.
    const directionalLight = new THREE.DirectionalLight(0xffffff, 1);
    directionalLight.position.set(200, 200, 200);
    directionalLight.castShadow = true;
    directionalLight.shadow.camera.near = 0.01;
    directionalLight.shadow.camera.far = 500;
    directionalLight.shadow.camera.right = 30;
    directionalLight.shadow.camera.left = -30;
    directionalLight.shadow.camera.top = 30;
    directionalLight.shadow.camera.bottom = -30;
    directionalLight.shadow.mapSize.width = 1024;
    directionalLight.shadow.mapSize.height = 1024;
    directionalLight.shadow.radius = 4;
    directionalLight.shadow.bias = -0.00006;
    // directionalLight.intensity = 200;

    scene.add(directionalLight);
    // 添加方向光助手
    const directionalLightHelper = new THREE.DirectionalLightHelper(directionalLight, 6);
    scene.add(directionalLightHelper);
    // 半球光.
    const fillLight = new THREE.HemisphereLight(0x8dc1de, 0x00668d, 1);
    fillLight.position.set(2, 1, 1);
    scene.add(fillLight);

    loader.load(
        // 模型路径
        '/assets/gltfs/ZhanTing625/SM_ZhanTing625.gltf',
        // 加载完成的回调函数
        function (gltf) {
            scene.add(gltf.scene);
        },
        // 加载过程中的回调函数
        function (xhr) {
            console.log((xhr.loaded / xhr.total) * 100 + '% 已加载');
        },
        // 加载出错的回调函数
        function (error) {
            console.error('加载模型出错', error);
        }
    );

    // 渲染循环
    function animate() {
        controls.update();
        renderer.render(scene, camera);
    }

};

[Jinxishihenian]

Thanks for the reply, but the sad problem remains, the second warning has been dealt with, I think the first warning is normal because we are considering browsers that do not support WebGPURenderer. WebGPURenderer:

WebGLRenderer:

I've output using different renderer vertex UV coordinates, and the two are exactly the same, and texture.repeat is also 1, and it doesn't shrink or zoom in, which confuses me,.

[Mugen87]

Okay, I can reproduce the issue. The asset is correctly rendered in the three.js editor. At least the result is similar to other viewers e.g. https://sandbox.babylonjs.com/.

When using WebGPURenderer, textures are messed up probably due to wrong texture coordinates. Both the WebGPU and WebGL backends are affected.

[Mugen87]

It seems there is an issue with the AO map. Can you confirm that adding this code snippet in your onLoad() callback of GLTFLoader solves the issue:

gltf.scene.traverse( object => {

    if ( object.isMesh ) object.material.aoMap = null;

} );

This removes the aoMap from the asset for testing.

[Mugen87]

@sunag I've tried to investigate the issue more closely but got stuck with a strange problem. The asset uses an AO map for its materials with channel 1. The diffuse and normal map use channel 0. When the AO map is removed, the asset renders as expected. With the AO map, it seems the texture is incorrectly sampled. It's even more strange that when refreshing the page, the AO map might get sampled differently.

It seems the instance of AONode is correctly configured with uv1 as the UVNodes data source. But something after this is broken. Could this issue be related to some node cache mechanism?

You can copy/paste below code in webgpu_loader_gltf for debugging. The test asset from https://github.com/mrdoob/three.js/issues/28758#issuecomment-2198841940 should be placed in examples/models/gltf/test.

<!DOCTYPE html>
<html lang="en">
    <head>
        <title>three.js webgpu - gltf loader</title>
        <meta charset="utf-8">
        <meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0">
        <link type="text/css" rel="stylesheet" href="main.css">
    </head>
    <body>

        <div id="info">
            <a href="https://threejs.org" target="_blank" rel="noopener">three.js</a> webgpu - gltf loader<br />
            Battle Damaged Sci-fi Helmet by
            <a href="https://sketchfab.com/theblueturtle_" target="_blank" rel="noopener">theblueturtle_</a><br />
            <a href="https://hdrihaven.com/hdri/?h=royal_esplanade" target="_blank" rel="noopener">Royal Esplanade</a> by <a href="https://hdrihaven.com/" target="_blank" rel="noopener">HDRI Haven</a>
        </div>

        <script type="importmap">
            {
                "imports": {
                    "three": "../build/three.webgpu.js",
                    "three/tsl": "../build/three.webgpu.js",
                    "three/addons/": "./jsm/"
                }
            }
        </script>

        <script type="module">

            import * as THREE from 'three';

            import { RGBELoader } from 'three/addons/loaders/RGBELoader.js';

            import { OrbitControls } from 'three/addons/controls/OrbitControls.js';
            import { GLTFLoader } from 'three/addons/loaders/GLTFLoader.js';

            let camera, scene, renderer;

            init();

            async function init() {

                const container = document.createElement( 'div' );
                document.body.appendChild( container );

                camera = new THREE.PerspectiveCamera( 45, window.innerWidth / window.innerHeight, 0.25, 20 );
                camera.position.set( - 1.8, 0.6, 2.7 );

                scene = new THREE.Scene();

                const rgbeLoader = new RGBELoader().setPath( 'textures/equirectangular/' );
                const texture = await rgbeLoader.loadAsync( 'royal_esplanade_1k.hdr' );

                texture.mapping = THREE.EquirectangularReflectionMapping;

                scene.background = texture;
                scene.environment = texture;

                // model

                const loader = new GLTFLoader().setPath( 'models/gltf/test/' );
                const gltf = await loader.loadAsync( 'msg.gltf' );
                scene.add( gltf.scene );

                // gltf.scene.traverse( object => {

                //  if ( object.isMesh ) {

                //      object.material.aoMap = null;

                //  }

                // } );

                renderer = new THREE.WebGPURenderer( { antialias: true } );
                renderer.setPixelRatio( window.devicePixelRatio );
                renderer.setSize( window.innerWidth, window.innerHeight );
                renderer.setAnimationLoop( animate );
                renderer.toneMapping = THREE.ACESFilmicToneMapping;
                container.appendChild( renderer.domElement );

                const controls = new OrbitControls( camera, renderer.domElement );
                controls.minDistance = 2;
                controls.maxDistance = 10;
                controls.target.set( 0, 0, - 0.2 );
                controls.update();

                window.addEventListener( 'resize', onWindowResize );

            }

            function onWindowResize() {

                camera.aspect = window.innerWidth / window.innerHeight;
                camera.updateProjectionMatrix();

                renderer.setSize( window.innerWidth, window.innerHeight );

            }

            //

            function animate() {

                renderer.render( scene, camera );

            }

        </script>

    </body>
</html>

[Jinxishihenian]

It seems there is an issue with the AO map. Can you confirm that adding this code snippet in your onLoad() callback of GLTFLoader solves the issue:

gltf.scene.traverse( object => {

    if ( object.isMesh ) object.material.aoMap = null;

} );

This removes the aoMap from the asset for testing.

This really solves the problem with this model(https://github.com/mtsee/vr-hall/tree/main/public/assets/room1),. Unfortunately, the issue of GLTF for the company's assets was not resolved,Sorry I used to think that the model was caused by a problem with my model and now it doesn't seem to be. However, I extracted the problematic gltf in the following compressed file iss.zip

WebGPURenderer（Wrong chair texture）：

WebGLRenderer（Correct chair texture）：

After I constantly tweaked the model, I found an important thing, as if the wooden ceiling at the top affected the texture of the chair at the bottom. When I remove the wooden ceiling at the top,The chair's texture map turned out to be working correctly.

WebGPURenderer（Correct chair texture）：

[sunag]

TextureNode will honor this.value.channel (texture.channel), for some reason albedo is returning channel=0 here:

The texture that I identified with a UV problem is albedo. If I force 'channel=1' it's work as expected.

[Jinxishihenian]

I tried setting 'channel=1' but then the texture of the ceiling started to work abnormally. Is this due to a gltf file error?

[sunag]

I think it's caused by the KHR_texture_transform extension and somehow the uv transform matrix is ​​being shared, I have to check this more carefully.

[Jinxishihenian]

As you said, I did enable KHR_texture_transform when exporting,It is a gltf export plugin for 3Dmax.

Related information:

3DMAX: 3DMAX version 2019

Export plugins:

[sunag]

This PR should fix this problem.

[Jinxishihenian]

Thank you very much