Our 3D avatar control can currently only control Mixamo avatars, and does so very poorly.
Controlling more general avatars (especially with faces) is necessary.
Description
We should add a rule-based 3D avatar solver such as https://github.com/yeemachine/kalidokit to be solve the avatar positions from the mediapipe poses directly.
Alternatives
Neural-network only approach. This will require training a better avatar model.
Problem
Our 3D avatar control can currently only control Mixamo avatars, and does so very poorly. Controlling more general avatars (especially with faces) is necessary.
Description
We should add a rule-based 3D avatar solver such as https://github.com/yeemachine/kalidokit to be solve the avatar positions from the mediapipe poses directly.
Alternatives
Neural-network only approach. This will require training a better avatar model.