search

google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://ai.google.dev/edge/mediapipe
Apache License 2.0
26.82k stars 5.09k forks source link

How to improve leg's accuracy? #4969

Open fchen09 opened 10 months ago

fchen09 commented 10 months ago

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

Yes

OS Platform and Distribution

ios 17.1

MediaPipe Tasks SDK version

0.10.8

Task name (e.g. Image classification, Gesture recognition etc.)

Pose Landmark Detection

Programming Language and version (e.g. C++, Python, Java)

swift

Describe the actual behavior

the tracking of rear leg is wonky

Describe the expected behaviour

the tracking of rear leg should be steady

Standalone code/steps you may have used to try to get what you need

I followed the link to build an iOS framework.
https://developers.google.com/mediapipe/framework/getting_started/hello_world_ios

And its performance is like the video showed.
https://drive.google.com/file/d/1VA9YBQNmhanmj1zbjZd_ilQZb29alPy1/view?usp=sharing

You can see the rear leg is wonky and unsteady, especially when I face toward the right.
How can I improve the rear-leg tracking?
Thank you!

Other info / Complete Logs

No response

kuaashish commented 10 months ago

Hi @schmidt-sebastian,

This pertains to our legacy pose detection and performance concerns. Currently, there are no tutorials available for our new Pose Landmaker on iOS. We would appreciate any suggestions you may have to address this matter.

Thank you

fchen09 commented 10 months ago

@schmidt-sebastian @kuaashish I appreciate your help in advance. I am curious whether the new Pose landmaker performs better than the legacy since it uses the same ML model.

schmidt-sebastian commented 9 months ago

There are some differences in pre- and post-processing as well as potential landmark smoothing. We don't have an in-depth analysis though so you would have to perform your own comparison for now.