search

sign-language-processing / pose

Library for viewing, augmenting, and handling .pose files
https://pose-format.readthedocs.io/en/latest/
MIT License
78 stars 23 forks source link

Handling Jittery and Noisy Pose Data - Especially Hand Landmarks #100

Open keniel123 opened 3 months ago

keniel123 commented 3 months ago

Hi there,

First, I'd like to thank you for the fantastic work on this package. I've been processing video data using pose estimations from both MediaPipe and OpenPose, and while these tools generate good results, I frequently encounter issues with jittery and noisy data over time. This is particularly challenging when dealing with hand landmarks, where the data tends to be less stable and more prone to fluctuations throughout the video.

I noticed that your package includes a normalization function, and I was wondering if this function could be used to mitigate some of the jitter and noise in the pose data extracted from video frames. If so, could you provide some guidance on how best to apply it? Alternatively, if you have other recommendations for dealing with this issue in video processing, I'd greatly appreciate your advice.

Looking forward to your suggestions and any potential updates to the documentation that could help with this.

Thank you!

AmitMY commented 3 months ago

Hi! there is no jitter normalization function. If you want to work with poses, and get more consistent results, consider post-processing using something like https://omnimotion.github.io/ - I came up with "how" (see https://github.com/qianqianwang68/omnimotion/issues/5) but never implemented it