Animation support?

[Benjythebee]

Heya, I noted your repo was slightly more active than the original. Would you be keem in adding animation group support? or is that too hard?

Cheers

[Benjythebee]

After some lookup on threejs I found this code;

export function loadMixamoAnimation( url, vrm ) {

    const loader = new FBXLoader(); // A loader which loads FBX
    return loader.loadAsync( url ).then( ( asset ) => {

        const clip = THREE.AnimationClip.findByName( asset.animations, 'mixamo.com' ); // extract the AnimationClip

        const tracks = []; // KeyframeTracks compatible with VRM will be added here

        const restRotationInverse = new THREE.Quaternion();
        const parentRestWorldRotation = new THREE.Quaternion();
        const _quatA = new THREE.Quaternion();
        const _vec3 = new THREE.Vector3();

        // Adjust with reference to hips height.
        const motionHipsHeight = asset.getObjectByName( 'mixamorigHips' ).position.y;
        const vrmHipsY = vrm.humanoid?.getNormalizedBoneNode( 'hips' ).getWorldPosition( _vec3 ).y;
        const vrmRootY = vrm.scene.getWorldPosition( _vec3 ).y;
        const vrmHipsHeight = Math.abs( vrmHipsY - vrmRootY );
        const hipsPositionScale = vrmHipsHeight / motionHipsHeight;

        clip.tracks.forEach( ( track ) => {

            // Convert each tracks for VRM use, and push to `tracks`
            const trackSplitted = track.name.split( '.' );
            const mixamoRigName = trackSplitted[ 0 ];
            const vrmBoneName = mixamoVRMRigMap[ mixamoRigName ];
            const vrmNodeName = vrm.humanoid?.getNormalizedBoneNode( vrmBoneName )?.name;
            const mixamoRigNode = asset.getObjectByName( mixamoRigName );

            if ( vrmNodeName != null ) {

                const propertyName = trackSplitted[ 1 ];

                // Store rotations of rest-pose.
                mixamoRigNode.getWorldQuaternion( restRotationInverse ).invert();
                mixamoRigNode.parent.getWorldQuaternion( parentRestWorldRotation );

                if ( track instanceof THREE.QuaternionKeyframeTrack ) {

                    // Retarget rotation of mixamoRig to NormalizedBone.
                    for ( let i = 0; i < track.values.length; i += 4 ) {

                        const flatQuaternion = track.values.slice( i, i + 4 );

                        _quatA.fromArray( flatQuaternion );

                        // 親のレスト時ワールド回転 * トラックの回転 * レスト時ワールド回転の逆
                        _quatA
                            .premultiply( parentRestWorldRotation )
                            .multiply( restRotationInverse );

                        _quatA.toArray( flatQuaternion );

                        flatQuaternion.forEach( ( v, index ) => {

                            track.values[ index + i ] = v;

                        } );

                    }

                    tracks.push(
                        new THREE.QuaternionKeyframeTrack(
                            `${vrmNodeName}.${propertyName}`,
                            track.times,
                            track.values.map( ( v, i ) => ( vrm.meta?.metaVersion === '0' && i % 2 === 0 ? - v : v ) ),
                        ),
                    );

                } else if ( track instanceof THREE.VectorKeyframeTrack ) {

                    const value = track.values.map( ( v, i ) => ( vrm.meta?.metaVersion === '0' && i % 3 !== 1 ? - v : v ) * hipsPositionScale );
                    tracks.push( new THREE.VectorKeyframeTrack( `${vrmNodeName}.${propertyName}`, track.times, value ) );

                }

            }

        } );

        return new THREE.AnimationClip( 'vrmAnimation', clip.duration, tracks );

    } );

}

https://pixiv.github.io/three-vrm/packages/three-vrm/examples/humanoidAnimation/index.html

I wonder if it's something we can reproduce on BBjs and VRM-loader

[spencer17x]

First of all, my plan is to extract babylonjs-vrm-loader from the repo as a separate monorepo. Then, compare three-vrm and try my best to complete the missing functions.I will notify you if there are any changes.

[spencer17x]

Or you can also use 1.x vrm, which supports vrm animation，If it's anything like threejs, it might take a lot of refactoring