Loading different animation resource from same skeleton, different file causes corrupt animation (GLTF)

[stryker313]

Godot version

v3.3.3.stable.official

System information

Windows 10 GTX 1060 6gb Intel 4690k

Issue description

Potentially linked issues include #50212, but perhaps less so #51785.

50212 Doesn't focus on loading the resource.

You can have two of the same skeletons, no difference in settings; however, when you attempt to move animations between these skeletons, the animation is corrupted.

ex. Skeleton A has animation B and Skeleton C has animation D. A and C are the same skeleton, no difference in anything, except for the fact they are two different exports.

You can't copy animation D to skeleton A. The animation ends up being corrupted/off. Same goes for animation B to Skeleton D.

It doesn't matter if you copy paste the animations, save the animations as a resource then load them into different skeletons, or turn off optimize animations in the GLTF import.

Here is short vid: https://youtu.be/Y6JX2CK95d8

Steps to reproduce

Export two armatures/skeletons from blender. Same skeleton, don't change anything, just give both of these skeletons different animations.

Load these armatures into godot, and create a scene for each GLTF import.

Copy the animation from one skeleton onto the other one, and the animation will not be correct, despite there being no difference in the skeletons.

Minimal reproduction project

There are two models. INSPECTAHDECK and METHODMAN. For this example, copy the animation "Armature002|mixamocom|Layer0 Retarget" from INSPECTAHDECK to METHODMAN, either by copy paste or saving and loading a resource. The animation will come out corrupted. In scene METHODMAN, this has already been done as an example. The result animation is labeled "CorruptAnimationExample", however for debugging purposes you may want to follow the above instructions.

Regardless of models and animations, this issue plagues all models and anims. Different animations using the same skeleton won't work correctly unless they are exported in the same file

Ignore the blockBroken animation, that one works on both because it was exported alongside both skeletons, so it has no impact on the bug/investigation AnimationTransferBug.zip

[fire]

My analysis of similar problems was done in https://github.com/godotengine/godot-proposals/issues/2961 and https://github.com/godotengine/godot/pull/51057.

Godot does not have generalized retargeting support and glTF2 if you are not strict about starting from the rest pose has an issue with reset.

Edited:

Master is broken so I will have some trouble debugging.

[stryker313]

I'll give you as much data as I can, and create more debug projects if you need them, as I have run into this problem quite frequently across many different scenarios. I use rokoko retargeting tool to put animations on my skeleton What's weird is that I verify that the supposed "rest_pose" (in blender, there is the "pose_position" and "rest_position") are the same, or at least appear to be before the files are exported. Once again, nothing modified, nothing change, just different animations with the same rest pose. So, at least from my perspective, something is either happening during the GLTF export from blender or during the GLTF import by godot that is causing some type of shift, because if the animations can work when exported WITH the skeleton, but not work when exported WITHOUT the skeleton, something has to be occurring during the transition from blender to gltf or gltf to godot.

A workaround for some people can be https://github.com/smix8/GodotAnimationRetargeting, however this doesn't work 100% and minor repair work is needed after using it.

[fire]

Here's my copy of the retargeter from rokoko, may the Blender addon help. https://github.com/V-Sekai/blender-skeletal-motion-animate

[fire]

Also please investigate the "fortune" and "temperance" algorithms from Blender. The fortune and temperance algorithms happen on Blender import.

[stryker313]

Im confused by the blender addon. How is that one different from the one you can get from https://github.com/Rokoko/rokoko-studio-live-blender ? Will look into fortune and temperance

[fire]

@stryker313 I want to streamline using the Blender plugin and making it more usable.

[stryker313]

Currently looking into fortune and temperance on GLTF import, but something to note: This error happens on the SAME skeleton, SAME save file, making me believe that import error is less likely.

For example, if you import a gltf, save the file.Then resuse this file. ie. retargeting anims to this skeleton, exporting, then not saving the skeleton so that the save file can be used a clean slate for other animations. I don't know that fortune and temperance would be contributing to this error if it is the same skeleton with the same rest pose (unless rokoko would be changing the rest pose, which it would seem is unlikely) Even though you are importing a different skeleton for retargeting animations, the target skeleton is still not changing, nor would the rest pose.

Are there any settings that you think could make a difference in the GLTF export? (That's a more likely scenario I'd think. Between that and godot)

[stryker313]

I found the solution. So this isn't a godot issue. What happens is that when you export a skeleton from Blender, whatever pose the skeleton is in at the moment of export becomes the "rest pose". Your skeleton does not inherit the rest position that is listed in the armature settings. So the simple solution is to make sure that when you export a gltf skeleton, that you make sure the skeleton is in its rest pose.

I think that it would be a good idea for Godot to potentially warn the user that their animation/gltf skeleton does not have the same rest pose when moving animations around like this, because this issue isn't something that is apparent. You would assume that the rest pose listed on the armature would be the rest pose that gets exported. Its vexed me for a bit, but I imagine that other people might run into a similar thing.

Your version of the rokoko addon, I imagine 1 of the steps should be to export the skeleton in rest pose, so that if a user accidentally forgets, its not a bother.

[fire]

Now is a good time remind you of the RESET proposal. https://github.com/godotengine/godot-proposals/issues/2961

You can make two defined RESET animations and then the pose the skeleton is in at the moment of export can be synced.

See docs https://docs.godotengine.org/en/stable/getting_started/workflow/assets/importing_scenes.html

It is important that the mesh is not deformed by bones when exporting. Make sure that the skeleton is reset to its T-pose or default rest pose before exporting with your favorite 3D editor.

[stryker313]

Nice. Retargeting wont be priority in the engine for a while if ever, but are there plans to ever incorporate a system like Unreal Engine (basically just built in retargeting)?

[fire]

I don't have a good technical approach to it.

Also if the issue is resolved, please close. Also you can find me on discord. I'll be around next week.

[TokageItLab]

Currently, if you want to set an optional T-pose as the initial pose for the model you are importing into the Godot, you need to sort on the T-pose animation into the top of animation tracks in the Blender.

@fire Is the cause of this problem that Blender's gltf exporter implicitly changes the skeleton of the glTF node (it will be Godot skeleton rest) depending on the order of the animation tracks? Or is it that Godot's glTF importer can't select the default pose animation?

If the cause is the second, how about adding the initial rest pose animation selector to the import settings?

However, if the cause is the first, we might want to send an issue/PR to Blender's I/O.

Edit: If you make an animation track named "RESET" on export, Godot 4 will use it as initial rest pose.

[fire]

Should be resolved by https://github.com/godotengine/godot/pull/53765.

Today's godot hash is https://github.com/godotengine/godot/commit/90a1e5193366e57069951cf2703616248e5f515d