debugging joint interpolation not updating

[kevzettler]

Hi @chinedufn,

Appreciate your thoughts on this. I have a setup where I have a modular character, and I'm trying to render 1-to-1 mesh to joint setup. I'm currently isolating the Head to get things working and go to the other body parts. Amazingly I have the Head mesh rendering and it looks correctly transformed for like the first keyframe of the animation. Screenshot below.

The problem I am having is that the joint data doesn't seem to update on further interpolateJoints calls. You can see the code I have instrumented below:

      animTime = (action.payload.elapsed + action.payload.dt) / 1000;

      interpolatedJoints = animationSystem.interpolateJoints({
        currentTime: animTime,
        jointNums: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19],
        currentAnimation: {
          keyframes: player.animation.actions.WalkCycle,
          startTime: 0,
          noLoop: false
        }
      });

     console.log(animTime, interpolatedJoints.joints[7]);

The console log output is in the screenshot above so you can see that the animTime is increasing, however, the joint output is not. Hopefully, I'm overlooking or misunderstanding something here.

[kevzettler]

Another piece to add that seems suspicious is:

interpolatedJoints.currentAnimationInfo

looks like:

{lowerKeyframeNumber: 0, upperKeyframeNumber: 1}

Not sure what these values are. lower/upper bounds on key frame indexes? Or pointers for Lower / Upper Quat keyframes?

[chinedufn]

Not sure what these values are. lower/upper bounds on key frame indexes? Or pointers for Lower / Upper Quat keyframes?

These are the indexes of the upper and lower keyframes. So if the keyframe times are 0, 1.555, 1.899 then keyframe number 0 and 1 are times 0 and 1.555. Have planned on revisiting this part of the API in the near-ish future though.

Anyways, back to your animation

---

First thing that comes to mind is that you might potentially be using the wrong joint.

I see that blender-actions-to-json README suggests that collada-dae-parser and blender-actions-to-json both export joints in the same order, but that isn't true (I thought it was when I was first writing the README.)

blender-actions-to-json is not guaranteed to use the same jointNameIndices as collada-dae-parser. An example of how I currently handle this is here https://github.com/chinedufn/blender-webgl-skinned-hot-reload-experiment/blob/master/viewer.js#L411-L412 .

I get the name of joint 0 from collada-dae-parser (which all your vertices are weighed against), then I look up that joint in blender-actions-to-json (which is handling your animation poses).

The quickest way that you can check if this is even the problem is

// Note the naming inconsistency... Work in progress...
console.log(parsedColladaModel.jointPositionIndices['Head']
console.log(blenderActions.jointNameIndices['Head']

If those are two different numbers then the above is likely the at least a part of the problem. It would mean that you think you're animating the head joints when you're really animating some other joints.

As always, making this more seamless is a work in progress and I'm always super happy for ideas / suggestions / PRs / etc

[chinedufn]

Another thing would be to either console.log(player.animation.actions.WalkCycle) or set a breakpoint inside of skeletal-animation-system

Check if the data there is what you'd expect.

So, find the first keyframe (lowest time) and the second keyframe (second lowest time). Are the numbers different? If they're the same then that would explain why all of your pose matrices are the same.

[kevzettler]

Ok first bit

The quickest way that you can check if this is even the problem is

// Note the naming inconsistency... Work in progress... console.log(parsedColladaModel.jointPositionIndices['Head'] console.log(blenderActions.jointNameIndices['Head']

I do not have access to a parsedColladaModel here. I am trying to directly use the skeleton from blender and bind dynamic meshes too it.

When logging the followin: console.log(blenderActions.jointNameIndices['Head']);

I get 7

Are you saying I can't trust that index is for the Head?

Another thing would be to either console.log(player.animation.actions.WalkCycle) or set a breakpoint inside of skeletal-animation-system

Check if the data there is what you'd expect.

Ok this definitely looks like an issue. A dump of the keyframes like:

      Object.keys(player.animation.actions.WalkCycle).forEach((key) => {
        console.log(player.animation.actions.WalkCycle[key][0]);
      });

Shows that they are all equal:

 [-1.780763586025245e-10, -5.721468006350502e-12, 4.654577425876866e-10, 1.0000000375293054, 3.8004191315804945e-17, 2.0485681691480823e-15, -6.209458766346692e-18, 2.1378705097508107e-26]
 [-1.780763586025245e-10, -5.721468006350502e-12, 4.654577425876866e-10, 1.0000000375293054, 3.8004191315804945e-17, 2.0485681691480823e-15, -6.209458766346692e-18, 2.1378705097508107e-26]
[-1.780763586025245e-10, -5.721468006350502e-12, 4.654577425876866e-10, 1.0000000375293054, 3.8004191315804945e-17, 2.0485681691480823e-15, -6.209458766346692e-18, 2.1378705097508107e-26]
[-1.780763586025245e-10, -5.721468006350502e-12, 4.654577425876866e-10, 1.0000000375293054, 3.8004191315804945e-17, 2.0485681691480823e-15, -6.209458766346692e-18, 2.1378705097508107e-26]
 [-1.780763586025245e-10, -5.721468006350502e-12, 4.654577425876866e-10, 1.0000000375293054, 3.8004191315804945e-17, 2.0485681691480823e-15, -6.209458766346692e-18, 2.1378705097508107e-26]
[-1.780763586025245e-10, -5.721468006350502e-12, 4.654577425876866e-10, 1.0000000375293054, 3.8004191315804945e-17, 2.0485681691480823e-15, -6.209458766346692e-18, 2.1378705097508107e-26]
[-1.780763586025245e-10, -5.721468006350502e-12, 4.654577425876866e-10, 1.0000000375293054, 3.8004191315804945e-17, 2.0485681691480823e-15, -6.209458766346692e-18, 2.1378705097508107e-26]

I will circle back to the original blender-actions data and see whats up. Thanks for all the help. This has been a good exercise for me to gain context on the modules to contribute PRs and docs in the future.

[chinedufn]

Are you saying I can't trust that index is for the Head?

Ohhhhh sorry, yes you can trust them then!

console.log(player.animation.actions.WalkCycle[key][0])

I'm assuming that [key][7] also has all of the same keyframes? Hmm yeah sounds like that might be the issue.

This has been a good exercise for me to gain context on the modules to contribute PRs and docs in the future.

Thanks for all of the digging! Pumped for those PRs and docs!

---