Distorted hair with facial expressions

alisaad673 commented 2 years ago

The hair get messed up while we use the facial expressions:

https://user-images.githubusercontent.com/35926530/181923684-4fe3b99c-6c77-4d56-9cc5-9a539568d5e5.mp4

jakezira commented 2 years ago

Several possibilities:

Hair might be directly affected by morphing
The backside of head mesh is modifying when morphing facial expressions, thus affecting the physics of hair

avaer commented 2 years ago

Someone needs to bisect this to find the bad commit. I suspect this regressed in the last month.

jakezira commented 2 years ago

The hair didn't work since scilly_drophunter_v31.6_Guilty.vrm is added --> https://github.com/webaverse/app/commit/917388d Same code works for all other avatar models, including the previous versions of same character. Seems like this 3d model has the problem.

jakezira commented 2 years ago

About the model

Model seems okay. It works on other 3rd party vrm viewers.

About the known issue of old threejs package

Tested on latest threejs (v143), and the problem still persist.

Test range

The issue happens on partial machines.

Working
- AMD Radeon RX 590
- AMD Radeon RX 5500 XT
- NVIDIA GeForce GTX 1660 Ti
Not working
- NVIDIA GTX 1080 Ti
- NVIDIA Quadro m5000m

jakezira commented 2 years ago

Wrote a test script, and hair works on threejs 143, failed on 137.4.

Here's index.html, cloned from blendshapes.html and avatar-debug.html of three-vrm examples which is used to test.

<!DOCTYPE html>

<html>
    <head>
        <meta charset="utf-8" />
        <title>three-vrm example</title>
        <meta
            name="viewport"
            content="width=device-width, initial-scale=1.0, minimum-scale=1.0, maximum-scale=1.0, user-scalable=no"
        />
        <style>
            body {
                margin: 0;
            }
            canvas {
                display: block;
            }
        </style>
    </head>

    <body>
        <script src="https://unpkg.com/three@0.143.0/build/three.js"></script>
        <script src="https://unpkg.com/three@0.143.0/examples/js/loaders/GLTFLoader.js"></script>
        <script src="https://unpkg.com/three@0.143.0/examples/js/controls/OrbitControls.js"></script> 
        <!--
        <script src="https://unpkg.com/three@0.137.4/build/three.js"></script>
        <script src="https://unpkg.com/three@0.137.4/examples/js/loaders/GLTFLoader.js"></script>
        <script src="https://unpkg.com/three@0.137.4/examples/js/controls/OrbitControls.js"></script> 
        -->
        <script src="./lib/three-vrm.js"></script>
        <script>
            // renderer
            const renderer = new THREE.WebGLRenderer();
            renderer.setSize( window.innerWidth, window.innerHeight );
            renderer.setPixelRatio( window.devicePixelRatio );
            document.body.appendChild( renderer.domElement ); // 137.4

            // camera
            const camera = new THREE.PerspectiveCamera( 30.0, window.innerWidth / window.innerHeight, 0.1, 20.0 );
            camera.position.set( 0.0, 1.0, 5.0 );

            // camera controls
            const controls = new THREE.OrbitControls( camera, renderer.domElement );
            controls.screenSpacePanning = true;
            controls.target.set( 0.0, 1.0, 0.0 );
            controls.update();

            // scene
            const scene = new THREE.Scene();

            // light
            const light = new THREE.DirectionalLight( 0xffffff );
            light.position.set( 1.0, 1.0, 1.0 ).normalize();
            scene.add( light );

            // gltf and vrm
            let currentVrm = undefined;
            const loader = new THREE.GLTFLoader();
            loader.crossOrigin = 'anonymous';
            loader.load(

                // URL of the VRM you want to load
                './models/scilly_drophunter_v31.6_Guilty.vrm',

                // called when the resource is loaded
                ( gltf ) => {

                    // generate VRM instance from gltf
                    THREE.VRMDebug.from( gltf ).then( ( vrm ) => {

                        console.log( vrm );
                        currentVrm = vrm;

                        scene.add( vrm.scene );

                        vrm.humanoid.getBoneNode( THREE.VRMSchema.HumanoidBoneName.Hips ).rotation.y = Math.PI;
                        vrm.springBoneManager.reset();

                    } );

                },

                // called while loading is progressing
                ( progress ) => console.log( 'Loading model...', 100.0 * ( progress.loaded / progress.total ), '%' ),

                // called when loading has errors
                ( error ) => console.error( error )

            );

            // helpers
            const gridHelper = new THREE.GridHelper( 10, 10 );
            scene.add( gridHelper );

            const axesHelper = new THREE.AxesHelper( 5 );
            scene.add( axesHelper );

            // update
            const clock = new THREE.Clock();
            clock.start();

            function animate() {

                requestAnimationFrame( animate );

                const deltaTime = clock.getDelta();

                if ( currentVrm ) {
                    // tweak blendshape
                    const s = Math.sin( Math.PI * clock.elapsedTime );
                    presets = [
                        THREE.VRMSchema.BlendShapePresetName.Angry,
                        THREE.VRMSchema.BlendShapePresetName.Fun,
                        THREE.VRMSchema.BlendShapePresetName.Joy,
                        THREE.VRMSchema.BlendShapePresetName.Sorrow,
                    ];
                    for( const preset of presets )  {
                        currentVrm.blendShapeProxy.setValue(
                            preset, s); 
                    }

                    // update vrm
                    currentVrm.update( clock.getDelta() );
                }
                renderer.render( scene, camera );

            }

            animate();
        </script>
    </body>
</html>

jakezira commented 2 years ago

Tried again with threejs 143 and three-vrm 0.6.11 on webaverse app and the hair rendering works.

webaverse / app