GLTFLoader: Broken animation with complex rig.

[donmccurdy]

This more complex rig (created with Rigify and exported from Blender 2.8) looks correct in Babylon but shows issues (stretching around the neck, for example) in three.js. I'd also expect the model to be upright, not horizontal, but that may be an unrelated issue.

rigify.zip

[donmccurdy]

Comparing this model in https://gltf.insimo.com/ and http://gltf-viewer.donmccurdy.com/, it seems to have worked correctly in r88, but no longer does as of r104 and probably earlier.

[Mugen87]

I've tested the model on the dev branch and I'm not able to reproduce the result from gltf-viewer. Instead, the animated model looks like loading it with Babylon.js or https://gltf.insimo.com/

My code:

var container, controls, clock;
var camera, scene, renderer, mixer;

init();
animate();

function init() {

    container = document.createElement( 'div' );
    document.body.appendChild( container );

    camera = new THREE.PerspectiveCamera( 45, window.innerWidth / window.innerHeight, 0.25, 20 );
    camera.position.set( - 1.8, 0.9, 2.7 );

    scene = new THREE.Scene();

    clock = new THREE.Clock();

    var loader = new THREE.RGBELoader().setPath( 'textures/equirectangular/' );
    loader.load( 'pedestrian_overpass_2k.hdr', function ( texture ) {

        texture.encoding = THREE.RGBEEncoding;
        texture.minFilter = THREE.NearestFilter;
        texture.magFilter = THREE.NearestFilter;
        texture.flipY = true;

        var cubeGenerator = new THREE.EquirectangularToCubeGenerator( texture, { resolution: 1024 } );
        cubeGenerator.update( renderer );

        var pmremGenerator = new THREE.PMREMGenerator( cubeGenerator.renderTarget.texture );
        pmremGenerator.update( renderer );

        var pmremCubeUVPacker = new THREE.PMREMCubeUVPacker( pmremGenerator.cubeLods );
        pmremCubeUVPacker.update( renderer );

        var envMap = pmremCubeUVPacker.CubeUVRenderTarget.texture;

        // model

        var loader = new THREE.GLTFLoader().setPath( 'models/gltf/' );
        loader.load( 'rigify.glb', function ( gltf ) {

            gltf.scene.traverse( function ( child ) {

                if ( child.isMesh ) {

                    child.material.envMap = envMap;

                }

            } );

            mixer = new THREE.AnimationMixer( gltf.scene );
            var action = mixer.clipAction( gltf.animations[ 0 ] );
            action.play();

            scene.add( gltf.scene );

        } );

        pmremGenerator.dispose();
        pmremCubeUVPacker.dispose();

        scene.background = cubeGenerator.renderTarget;

    } );

    renderer = new THREE.WebGLRenderer( { antialias: true } );
    renderer.setPixelRatio( window.devicePixelRatio );
    renderer.setSize( window.innerWidth, window.innerHeight );
    renderer.gammaOutput = true;
    container.appendChild( renderer.domElement );

    controls = new THREE.OrbitControls( camera, renderer.domElement );
    controls.target.set( 0, - 0.2, - 0.2 );
    controls.update();

    window.addEventListener( 'resize', onWindowResize, false );

}

function onWindowResize() {

    camera.aspect = window.innerWidth / window.innerHeight;
    camera.updateProjectionMatrix();

    renderer.setSize( window.innerWidth, window.innerHeight );

}

//

function animate() {

    requestAnimationFrame( animate );

    if ( mixer ) mixer.update( clock.getDelta() );

    renderer.render( scene, camera );

}

[Mugen87]

Closing for now. This seems to be an issue in the viewer application.

[donmccurdy]

Thanks! Tracking the viewer issue in https://github.com/donmccurdy/three-gltf-viewer/issues/150.

[donmccurdy]

@Mugen87 it seems that this behavior began in r103, when the invocation of .optimize() was moved out of clip constructors. My viewer optimizes each clip, and removing that fixes the problem. Here are more examples:

finder_rig.zip RobotExpressive.blend.zip

[donmccurdy]

I'm not sure why .optimize() is breaking these models. It won't work on keyframe tracks using glTF's cubic spline interpolation (known limitation), but even models using only linear interpolation seem affected here.

[Mugen87]

it seems that this behavior began in r103, when the invocation of .optimize() was moved out of clip constructors.

Are you referring to #14385? This change already lands in R95 🤔

[donmccurdy]

I didn't track down which change it was, but the problem does appear only in r103+ for me. #15893 looks relevant, but should only be affecting cubic spline animation, hm.

[SteveEdson]

Also had the same issue after updating from r102 -> r103 (still affected in r105). My animation was playing, but was moving in all the wrong places. Removing my .optimize() code fixes the issue.

[JordyvanDortmont]

I also encountered the same issue. The fix seems to be reverting the changes to these lines: https://github.com/mrdoob/three.js/pull/15893/commits/ec1d5a7a72f9afbf40c00c5db6b30d298820b860#diff-43f9265a3da0d9ebbf235517b2541578L2910-R2903

If reverted, optimize works in that specific commit as well as in r107.

I think there's some code in optimize that silently doesn't work with a TypedArray, or there could be an issue with one of the KeyFrameTrack contructors.

Should I open a PR or would you like to do some research first?

[donmccurdy]

@JordyvanDortmont could you open a new issue, with information to let us reproduce the problem? I wouldn't want to revert the changes without testing this against a few different cases.

If reverted, optimize works in that specific commit as well as in r107.

Do you mean reverting the entire commit, or just those two lines?

I think there's some code in optimize that silently doesn't work with a TypedArray, or there could be an issue with one of the KeyFrameTrack contructors.

TypedArray is supported, but optimize() doesn't work with tracks interpolated by the cubic-spline method, which some glTF files use. I'm not sure what the result would be if it were called in that case.

[JordyvanDortmont]

@donmccurdy sure! #17180

If reverted, optimize works in that specific commit as well as in r107.

Do you mean reverting the entire commit, or just those two lines?

Just those two lines. I tested reverting those and after that optimize worked.