Issue Summary:
During my experimentation with DynamicPose, I encountered significant challenges when retargeting poses to non-standard skeletons, particularly with images of cats. The retargeting algorithm fails to accurately map the shorter leg bones of cats, resulting in disproportionately elongated legs in the generated animations. This leads to unrealistic and undesired video outputs.
Detailed Description:
When attempting to retarget a pose from a human to a cat, the algorithm does not account for the anatomical differences between humans and cats, especially the shorter and differently proportioned limbs of cats. As a result, the legs in the retargeted animation appear excessively long and unnatural, compromising the overall quality and realism of the animation.
In contrast, I tested MusePose with the same setup, and their retargeting algorithm handled the pose mapping for cats much more effectively. MusePose successfully preserved the correct limb proportions
Expected Behavior:
The retargeting algorithm should accurately map the human pose to the cat's skeleton, maintaining realistic limb proportions. Specifically, the shorter legs of the cat should be preserved in the animation, ensuring the resulting video appears natural and faithful to the original anatomical structure of the cat.
Actual Behavior:
The retargeted pose results in disproportionately long legs for the cat, making the animation appear unrealistic and unnatural. The anatomical discrepancies between humans and cats are not adequately addressed, leading to distorted animations.
I appreciate the hard work and dedication of the DynamicPose development team. Addressing this issue will significantly enhance the tool's flexibility and applicability across a broader range of projects. Thank you for considering this feedback, and I look forward to any updates or improvements in future releases.
SlimeVRX
commented
2 months ago
Issue Summary: During my experimentation with DynamicPose, I encountered significant challenges when retargeting poses to non-standard skeletons, particularly with images of cats. The retargeting algorithm fails to accurately map the shorter leg bones of cats, resulting in disproportionately elongated legs in the generated animations. This leads to unrealistic and undesired video outputs.
Detailed Description: When attempting to retarget a pose from a human to a cat, the algorithm does not account for the anatomical differences between humans and cats, especially the shorter and differently proportioned limbs of cats. As a result, the legs in the retargeted animation appear excessively long and unnatural, compromising the overall quality and realism of the animation.
In contrast, I tested MusePose with the same setup, and their retargeting algorithm handled the pose mapping for cats much more effectively. MusePose successfully preserved the correct limb proportions
Retarget in MusePose:
https://github.com/user-attachments/assets/b13988ea-d0ec-4087-8c63-f55a9104997f
Comparison dance_1 (First video is MusePose, Second video is DynamicPose)
https://github.com/user-attachments/assets/0691a064-b17e-454b-aa8b-76c24a440d8c
https://github.com/user-attachments/assets/59c3d4fd-0cff-4f9c-b905-1f3367981628
Comparison dance_2 (First video is MusePose, Second video is DynamicPose)
https://github.com/user-attachments/assets/698c1254-bcb2-4246-b4d4-3d622324c3a0
https://github.com/user-attachments/assets/a5562db7-b67b-436b-ae7e-f8c6ff1261d7
Expected Behavior: The retargeting algorithm should accurately map the human pose to the cat's skeleton, maintaining realistic limb proportions. Specifically, the shorter legs of the cat should be preserved in the animation, ensuring the resulting video appears natural and faithful to the original anatomical structure of the cat.
Actual Behavior: The retargeted pose results in disproportionately long legs for the cat, making the animation appear unrealistic and unnatural. The anatomical discrepancies between humans and cats are not adequately addressed, leading to distorted animations.
I appreciate the hard work and dedication of the DynamicPose development team. Addressing this issue will significantly enhance the tool's flexibility and applicability across a broader range of projects. Thank you for considering this feedback, and I look forward to any updates or improvements in future releases.
Thank you!