search

google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://ai.google.dev/edge/mediapipe
Apache License 2.0
27.72k stars 5.18k forks source link

Pose landmarker (Heavy) model's pose world landmarks for head part seems distorted #5548

Open wxhzzsf opened 4 months ago

wxhzzsf commented 4 months ago

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

Yes

OS Platform and Distribution

iOS 17.5.1

Mobile device if the issue happens on mobile device

iPhone 15 Pro Max

Browser and version if the issue happens on browser

No response

Programming Language and version

Swift

MediaPipe version

0.10.14

Bazel version

No response

Solution

Pose

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

No response

Xcode & Tulsi version (if issue is related to building for iOS)

No response

Describe the actual behavior

Pose landmarker (Heavy) model's pose world landmarks for head part seems distorted

Describe the expected behaviour

The head part of the pose world landmarks shouldn't be distorted

Standalone code/steps you may have used to try to get what you need

I'm using ARkit to display the pose world landmarks for latest Pose landmarker (Heavy) model. All joints seems to be correct except world landmarks 0 - 10. Please check and update the model thanks!

Other info / Complete Logs

No response

kuaashish commented 4 months ago

Hi @wxhzzsf,

Could you please provide the complete steps you are following from our documentation? This will help us understand the issue better. If necessary, we will reproduce and investigate the issue in more detail.

Thank you!!

wxhzzsf commented 4 months ago

Hi @wxhzzsf,

Could you please provide the complete steps you are following from our documentation? This will help us understand the issue better. If necessary, we will reproduce and investigate the issue in more detail.

Thank you!!

Here is the iOS sample project I started working with. I started developing using the pose_landmarker_heavy.task model in the Xcode project. I rendered the 3D pose world landmarks in ARKit. The lite and full model seemed fine for the pose world landmarks but only the heavy model's pose world landmarks from 0 to 10 seemed distorted. Maybe you guys can render the heavy model's pose world landmarks in python and check whether it's working fine?

wxhzzsf commented 3 months ago

@kuaashish I'm actually using 0.10.14 version. 0.8.14 was a typo.