search

met4citizen / TalkingHead

Talking Head (3D): A JavaScript class for real-time lip-sync using Ready Player Me full-body 3D avatars.
MIT License
349 stars 107 forks source link

Poses and animations downloaded from Mixamo moves the origin of the model #23

Closed shengxijin closed 7 months ago

shengxijin commented 7 months ago

When using custom files downloaded from Mixamo, applying the .fbx poses/animations seems to shift the center of the model and fixes the model's hip bone on the floor instead of the feet. This only affects poses and animations downloaded from Mixamo; the included dancing.fbx file doesn't have this issue.

This shift affects both the default model and any other custom model downloaded from readyplayer.me

Below is "Female Standing" applied to the default brunette.glb

image

Any idea what's happening here? All the Mixamo files apply just fine in Blender or Unity.

PS: Great repository and work, by the way, this really cool stuff.

met4citizen commented 7 months ago

Thanks.

I just downloaded and tried the same Mixamo pose 'Female Standing Pose.fbx' with the test app, and I couldn't reproduce the error you are having (see before and after screenshots at the end).

The first thing you should perhaps try is to check that you are using either the 'latest' or 'v1.1' version from CDN, or if you have downloaded the source files, ensure you have all the latest versions.

If that doesn't help, have you edited the FBX file in either Blender or Unity? If you have, could this be some problem in export?

pose-before pose-after

shengxijin commented 7 months ago

I think I found what's causing it. Very early on in my workflow, I had used Blender to export brunette.glb to .fbx and uploaded it to Mixamo to see how poses looked on that model. Since there's no way to 'unload' a model from Mixamo, I tried creating a new account to download poses using the Mixamo "default character", and now the downloaded poses work.

Also, if you apply any of the stock characters found on the Mixamo library, the poses/animations break in other ways. For example, downloading this "Female Standing Pose" when used on the "Remy" character shifts the brunette.glb center of the model in the opposite direction; and the pose doesn't work at all if downloaded while applied to the "Suzie" character.

The core issue seems to be coming from Blender during export to .fbx, but I'm still working on figuring out what.

Thanks for the hint! And keep up the good work!