Issues with retargeting

[owlandfox]

I have setup my avatar according to documentation and all looks correct in the mapping -

Correct scripts have been added to character -

But the animation is completely out. Head is too high and not moving in sync with upper chest and shoulders, and hands are not rotating at the correct place. Wrist rotation seems to be happening higher up the lower arm.

[sohailshafiiWk]

Hello. It appears that your character might need IK constraints, specifically deformation constraint that attempts to preserve the character's constraints using Animation Rigging. After importing the package files (see my reply in https://github.com/oculus-samples/Unity-Movement/issues/35), please look at the prefab ArmatureSkinningUpdateRetargetUser used in MovementRetargeting.unity. You can add similar constraints by right-clicking on a character, then navigating to Movement Samples -> Body Tracking -> Animation Rigging Retargeting (constraints).

[owlandfox]

So even when building the retargeting scene to device (obviously without making any changes to it as I cant) the rig is completely out. Added a secondary robot via the UI with animation retargeting and you can see the mesh is completely deached from rig. Any idea what could be happening here?

[sohailshafiiWk]

Can you indicate which version of Unity you are using? There is currently a retargeting bug with 2022 and 2023 that is being investigated. See https://github.com/oculus-samples/Unity-Movement/issues/29

[owlandfox]

I downgraded the project back to 2021.3.30 and it did correct the sample scene and retargeted character. I re-applied all the constraints to my character to match the retargeted sample robot and its much better but still quite off as you can see my hands are super long and still twisting at a strange place on the wrist. Neck is at a strange height above the shoulders too. Any advice to further refine this much appreciated -

Can you indicate which version of Unity you are using? There is currently a retargeting bug with 2022 and 2023 that is being investigated. See #29

[sohailshafiiWk]

We are currently looking into a modified deformation system that might be able to help. For now, you can possibly use _jointRotationTweaks on RetargetingLayer to manually adjust bones on the humanoid.

[owlandfox]

In terms of getting the mesh and rig to perform/deform as well as the high fidelity character would we need to match the exact pose/bone length/rotations as the source character we are retargeting from. If so, which model are we actually retargeting from when applying the constraints from the samples scripts?

[sohailshafiiWk]

OVRUnityHumanoidSkeletonRetargeter and the class that inherits from it, RetargetingLayer, effectively retarget from OVRSkeleton's Bones to the humanoid bones. Note that OVRUnityHumanoidSkeletonRetargeter inherits from OVRSkeleton. The OVRSkeleton also gives the body tracking rest pose in the BindPoses variable.

[andkim-meta]

Hello. We have some updated documentation on setting up a retargeted character here that should help resolve the issue: Setting up a character for body tracking. Closing this issue for now.

[karunateam]

It doesn't seem that the suggested answer of following the new documentation is sufficient to close out this ticket. @owlandfox Can you confirm that following the new documentation fixed your issue?