search

google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://ai.google.dev/edge/mediapipe
Apache License 2.0
27.42k stars 5.15k forks source link

Pose landmarkers obtained with pose landmarker model are better than pose landmarkers obtained with holistic model #5569

Open neutronengas opened 2 months ago

neutronengas commented 2 months ago

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

Yes

OS Platform and Distribution

Linux Ubuntu 22.04

MediaPipe Tasks SDK version

0.10.14

Task name (e.g. Image classification, Gesture recognition etc.)

Pose / Holistic landmarker detection

Programming Language and version (e.g. C++, Python, Java)

Python

Describe the actual behavior

Pose landmarkers obtained with the model pose_landmarker_heavy.task are significantly more accurate than the pose landmarkers obtained with the model holistic_landmarker.task

Describe the expected behaviour

Pose landmarkers should be similar for the two models

Standalone code/steps you may have used to try to get what you need

Link to the colab: https://colab.research.google.com/gist/neutronengas/61d75d00238d2f4e93a932b4cd3360ef/-mediapipe_python_tasks-_pose_landmarker.ipynb

Other info / Complete Logs

No response

neutronengas commented 2 months ago

Update: i have realized that the output of the pose model coincides exactly with the output of the holistic model when replacing pose_landmarker_heavy.task with pose_landmarker_lite.task. I thus assume the holistic model to incorporate the lite pose model. Is there any way to obtain a version of the holistic model which incorporates the heavy pose landmarker model? Additionally, are there any options to smooth landmarks, comparable to the smoothing options in previous mediapipe versions?

Thank you very much in advance!

ChristianNSchmitz commented 2 months ago

We would be also very interested in these features!