AnimancerJob does not set correct bone rotation on some bones

[TigerHix]

Environment

• Animancer Version Number: 7.2
• Animancer Pro or Lite: Pro
• Unity Version: Unity 2021.3.7f1
• Platform: Windows

Description

I am creating a character animation software and use AnimancerJob to update my character's bone rotations. However, I found that on some models the rotations are not correctly applied. See below code:

            // In the Job struct derived from IAnimationJob
            public void ProcessAnimation(AnimationStream stream) {
                for (var i = 0; i < (int)HumanBodyBones.LastBone; i++) {
                    var boneTransform = Bones[i];
                    if (boneTransform.IsValid(stream)) {
                        var newRotation = Quaternion.SlerpUnclamped(boneTransform.GetLocalRotation(stream), OverrideBoneRotations[i], OverrideBoneRotationWeights[i]);
                        var offsetRotation = newRotation * BoneRotationOffsets[i] * AdditionalBoneRotationOffsets[i];

                        if (i == (int)HumanBodyBones.RightUpperArm) {
                            Debug.Log("Original rotation: " + boneTransform.GetLocalRotation(stream).eulerAngles);
                            Debug.Log("Override: " + OverrideBoneRotations[i].eulerAngles);
                            Debug.Log("Override weight: " + OverrideBoneRotationWeights[i]);
                            Debug.Log("Offset: " + BoneRotationOffsets[i].eulerAngles);
                            Debug.Log("Additional: " + AdditionalBoneRotationOffsets[i].eulerAngles);
                            Debug.Log("New rotation: " + newRotation.eulerAngles);
                        }

                        boneTransform.SetLocalRotation(stream, offsetRotation);

                        FinalBoneWorldPositions[i] = boneTransform.GetPosition(stream);
                        FinalBoneWorldRotations[i] = boneTransform.GetRotation(stream);
                        FinalBonePositions[i] = boneTransform.GetLocalPosition(stream);
                        FinalBoneRotations[i] = boneTransform.GetLocalRotation(stream);

                        if (i == (int)HumanBodyBones.RightUpperArm) {
                            Debug.Log("Final rotation: " + FinalBoneRotations[i].eulerAngles);
                        }
                    }
                }
            }

Console output:

As you can see, the actual rotation differs about ~20 degrees from the console output. However, if I do not use AnimancerJob, and instead manually set the bone transforms by animator.GetBoneTransform(bone).localRotation = ... in LateUpdate(), the problem is gone!

Would you know what's going on? I can provide the character model in question via email.

[KybernetikGames]

I haven't used animation jobs much, but I can't see anything obviously wrong with your code.

See if you can narrow down the cause a bit:

• Does it still happen if you ignore every bone other than RightUpperArm?
• Does it still happen if you only set the local rotation to some value based on i and don't do anything else?

[TigerHix]

@KybernetikGames Thanks for looking. I have tried your suggestions:

• It still happens if I just set the RightUpperArm bone.
• Yes. In fact even I set Quaternion.identity for every bone, some bones just would not have (0, 0, 0) rotation.

I found that this offset in bone rotations is different in every model. Could it be related to the model's bind pose, etc.?

[TigerHix]

Following shows comparison setting every bone rotation to (i, i, i):

AnimationJob:

animator.GetBoneTransform():

As you can see the differences are nontrivial here :(

[KybernetikGames]

Are you able to replicate the issue in the Simple Lean example scene (with the default character or yours)? I tried putting bone.SetLocalRotation(stream, Quaternion.identity); in its Job and it shows identity for those bones in the Inspector as expected, but the character goes into their T pose so it could just be the way the DefaultHumanoid is rigged that lets it work.

If you can email me a minimal reproduction project I'll take a look at it, but most likely all I'll be able to do is report a Unity bug because Animancer doesn't have any contact with the data between your job and what gets applied to the model.

[TigerHix]

Are you able to replicate the issue in the Simple Lean example scene (with the default character or yours)? I tried putting bone.SetLocalRotation(stream, Quaternion.identity); in its Job and it shows identity for those bones in the Inspector as expected, but the character goes into their T pose so it could just be the way the DefaultHumanoid is rigged that lets it work.

If you can email me a minimal reproduction project I'll take a look at it, but most likely all I'll be able to do is report a Unity bug because Animancer doesn't have any contact with the data between your job and what gets applied to the model.

Can you try a non-identity rotation like (10, 20, 30) on your end? I just tried modifying the SimpleLean.GetDefaultHumanoidLeanBones to return the two upper arm bones, and then in Job.ProcessAnimation I do bone.SetLocalRotation(stream, Quaternion.Euler(10, 20, 30));. The inspector shows mind-boggling values:

I also used the Animancer's character model to test. Here's the results:

[KybernetikGames]

Yep, I see the inconsistency with non-identity values and I've submitted a Unity bug report.

I found that setting the local rotation of the RightArm and LeftArm gave almost the correct values on LeftArm but RightArm was quite a bit off. Then if I added RightForeArm to it (the child of RightArm), somehow it causes RightArm to get a different rotation. And adding LeftUpLeg gets completely ignored by the job and just plays the animation normally.

Looks like you'll need to fall back to using LateUpdate.

If you have lots of Transforms to control in one go, I'd recommend using IJobParallelForTransform which is similar to Animation Jobs, but entirely outside of the animation system. I use it in FlexiMotion and the performance is amazing compared to regular scripts.

[TigerHix]

Yep, I see the inconsistency with non-identity values and I've submitted a Unity bug report.

I found that setting the local rotation of the RightArm and LeftArm gave almost the correct values on LeftArm but RightArm was quite a bit off. Then if I added RightForeArm to it (the child of RightArm), somehow it causes RightArm to get a different rotation. And adding LeftUpLeg gets completely ignored by the job and just plays the animation normally.

Looks like you'll need to fall back to using LateUpdate.

If you have lots of Transforms to control in one go, I'd recommend using IJobParallelForTransform which is similar to Animation Jobs, but entirely outside of the animation system. I use it in FlexiMotion and the performance is amazing compared to regular scripts.

Thank you so much, I will look into it. When Unity triages your bug report, could you link the issue tracker URL here? That'd be much appreciated. 😄

For now, I will stay with animation jobs because (unfortunately) my whole animation system is built around it. But I would read the FinalBonePositions and FinalBoneRotations (resulted from the animation job, which somehow contain the correct values) in LateUpdate to override the bones again. Hopefully this would work, at least for now.

Side rant: This is arguably the most essential functionality of Animation Jobs, and I can't believe no one else have filed a bug yet... Guess no one's really using animation jobs...

[KybernetikGames]

When Unity triages your bug report, could you link the issue tracker URL here?

Will do.

For now, I will stay with animation jobs because (unfortunately) my whole animation system is built around it. But I would read the FinalBonePositions and FinalBoneRotations

You could still do all the calculations in the animation job and just give those arrays to an IJobParallelForTransform job in LateUpdate to apply them.

[KybernetikGames]

This issue has been confirmed as a bug in all current Unity versions. Here's the public issue tracker link: https://issuetracker.unity3d.com/product/unity/issues/guid/UUM-27271

[TigerHix]

@KybernetikGames Unity has replied that this is by design on the issue page:

The results of this scene are to be expected. The imprecisions are also expected when using the Humanoid solve, as the Humanoid works in Muscle space values and conversion is required. Humanoid is not a lossless process and should only be used if retargeting is required. In order to avoid the these problematics completely, we recommend using the Generic Animation Type instead. The Generic type will give you the exact result you where expecting in this scene.

If you require Humanoid for retargeting purposes, here's an explanation of the results you are seeing. First, what you are basically doing is, playing a Humanoid Muscle clip, locking Euler values on some bones, and applying IK on the feet (default Humanoid behaviour). When using a Humanoid solve, Euler curves go through a conversion to Quaternions in Muscle space. When these values are converted back into transform values, they have the same visual orientation, but values may change and we cannot guarantee these values will be the same. Other Humanoid solve rules such as limits may also change these values. In this case, you are also applying IK to the feet (default Humanoid behaviour) after the clip and the set rotation, explaining why the foot still follows the IK goal and moves the leg (with the static value now moving to the IK). If you want to set the bone rotation in the Humanoid context, we recommend setting the Muscle values in Humanoid directly. This will avoid extra conversion and imprecision . You should still not expect exact values as the Humanoid solve will do a final conversion to transforms and this will not be lossless.

However, I don't think they are correctly evaluating the situation here. We repro'd the issue without playing an animation clip and using foot IK. Also, our workaround set the euler angles to the values we expected. What do you think?

[KybernetikGames]

I suspect they're correct in this case.

Whether an animation is playing or not doesn't matter, the animation job affects the animation stream which has to go through the Humanoid muscle system before being applied to the Transforms.

Setting the Transforms in LateUpdate doesn't have anything to do with animations/humanoid/muscles, it's just applying the values directly.