search

Flode-Labs / vid2densepose

Convert your videos to densepose and use it on MagicAnimate
MIT License
977 stars 126 forks source link

Inconsistency in DensePose Visualisation Across Consecutive Frames #19

Open yihong1120 opened 9 months ago

yihong1120 commented 9 months ago

Dear Vid2DensePose Maintainers,

I hope this message finds you well. I am reaching out to report an issue I've encountered while utilising the Vid2DensePose tool for a project aimed at enhancing animation sequences through detailed human pose estimation.

Issue Description: Upon generating the "Part Index" visualisations for a sequence of video frames, I observed an inconsistency in the visual output across consecutive frames. Despite the input video depicting minimal movement between frames, the DensePose visualisation appears to fluctuate, leading to a jittery and discontinuous representation of the pose data.

This inconsistency poses a challenge in achieving the desired temporal coherence for human image animation, particularly when integrating with MagicAnimate. The expected outcome is a smooth transition of pose visualisations that accurately reflect the movement within the video.

Steps to Reproduce:

  1. Follow the installation and usage guide as per the repository's README.md.
  2. Run the main.py script on a video with subtle movements.
  3. Observe the output video for any discrepancies in the DensePose visualisation across frames.

Potential Impact: The issue has the potential to affect users who rely on the tool for high-fidelity animation projects, where precision and consistency in human pose estimation are paramount.

Request: I kindly request your assistance in addressing this matter. Any insights into potential causes or suggestions for mitigating this inconsistency would be greatly appreciated. Additionally, if there are any recommended practices or parameters that could enhance the stability of the visual output, I would be eager to learn and implement them.

Acknowledgments: I would like to extend my gratitude for your efforts in developing such a valuable tool for the community. Your work significantly contributes to the advancement of human pose estimation applications.

Thank you for your time and support. I look forward to your response and any guidance you can provide.

Best regards, yihong1120

samsara-ku commented 3 months ago

Did you solve this problem? In my case, it is not still working well like below:

image image