Is there a way to prevent jittering during video pose estimation using DWPose?

[MoonEese]

Is there a way to prevent jittering during video pose estimation using DWPose? Frequent jittering occurs even when standing still.

[TZYSJTU]

Yes, I meet this problem, too. How ho make it smooth, are there any post-processing algrithms?

[ailingzengzzz]

Hi @MoonEese @TZYSJTU ,

Please refer to this repo: https://github.com/cure-lab/SmoothNet. This is a plug-and-play smooth model in the post-process stage. You can simply input the 2d pose sequences, and output the smooth sequences.

BTW, if you do not need to use a network, you can try simple filters: https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.savgol_filter.html. There are two parameters: window_length (you can try 7), polyorder (you can try 2).

[MoonEese]

@TZYSJTU I have used simple average filtering, and it was little bit simple and effective for me

@ailingzengzzz Thanks for great work and help!

[learnuser1]

@TZYSJTU I have used simple average filtering, and it was little bit simple and effective for me

@ailingzengzzz Thanks for great work and help! Does simple average filtering not result in blurred video？

[MoonEese]

@learnuser1 I used each coordinate from n, n+1, n+2, n+3, n+4 frame for averages, and start/end 2frames were original coordinates.

[learnuser1]

@MoonEese Thank you for your answer.

[ry555555]

@learnuser1 I used each coordinate from n, n+1, n+2, n+3, n+4 frame for averages, and start/end 2frames were original coordinates.

https://github.com/IDEA-Research/DWPose/assets/142297792/ec6ec85e-d07c-40dd-b9fd-da5c08afc2ac

@MoonEese My result has a bad ghosting. Could you provide the specific implementation?🙏

[MoonEese]

@ry555555

I can't view your result video, but I believe the term 'bad ghosting' arose because you averaged the frames of the DWPose result video itself. The method I mentioned earlier involves averaging the coordinate values extracted from DWPose, rather than directly averaging the result video frames. This means that the coordinate of, say, keypoint 1 in frame 1 is replaced by the average of keypoint 1's coordinates in frames 1, 2, 3, 4, 5. While this approach reduces jitter, it may cause issues with actions such as waving hands back and forth in greeting.

[ry555555]

@MoonEese Got it, Thx!