Have I written custom code (as opposed to using a stock example script provided in MediaPipe)
Yes
OS Platform and Distribution
Ubuntu 22.04.4 LTS
Mobile device if the issue happens on mobile device
No response
Browser and version if the issue happens on browser
No response
Programming Language and version
Python
MediaPipe version
0.10.13
Bazel version
No response
Solution
Pose
Android Studio, NDK, SDK versions (if issue is related to building in Android environment)
No response
Xcode & Tulsi version (if issue is related to building for iOS)
No response
Describe the actual behavior
NormalizedLandmark gives some values out of boundaries. The only difference between pose_landmarks and NormalizedLandmark is that the second option contains visibility and presence scores. I tried with latest/pose_landmarker_heavy.task and 1/pose_landmarker_heavy.task, they give the same results
Describe the expected behaviour
According to the documentation, Normalized Landmark represents a point in 3D space with x, y, z coordinates. x and y are normalized to [0.0, 1.0] by the image width and height respectively. z represents the landmark depth, and the smaller the value the closer the landmark is to the camera. I understand Mediapipe tries to make up the "invisible" joints, but wouldn't be better if it simply does not include them?
Standalone code/steps you may have used to try to get what you need
#!wget -O pose_landmarker.task -q https://storage.googleapis.com/mediapipe-models/pose_landmarker/pose_landmarker_heavy/float16/latest/pose_landmarker_heavy.task
from mediapipe import solutions
from mediapipe.framework.formats import landmark_pb2
import mediapipe as mp
from mediapipe.tasks import python
from mediapipe.tasks.python import vision
import cv2
import numpy as np
# STEP 1: Create an PoseLandmarker object.
base_options = python.BaseOptions(model_asset_path='pose_landmarker.task')
options = vision.PoseLandmarkerOptions(
base_options=base_options,
output_segmentation_masks=True)
detector = vision.PoseLandmarker.create_from_options(options)
# STEP 2: Load input images
frame = mp.Image.create_from_file('webcam1_erik3_cal_50.jpg') #RGB image, 638x478
detection_result = detector.detect(frame)
# Function to count landmarks
def count_landmarks(landmarks):
return [sum(1 for lm in landmarks if 0 < lm.x > 1), sum(1 for lm in landmarks if 0 < lm.y > 1)]
#analysis
pose_lm = count_landmarks(detection_result.pose_landmarks[0])
pose_norm_lm = count_landmarks([landmark_pb2.NormalizedLandmark(x=landmark.x, y=landmark.y, z=landmark.z) for landmark in detection_result.pose_landmarks[0]])
pose_landmarks_proto = landmark_pb2.NormalizedLandmarkList()
pose_landmarks_proto.landmark.extend([landmark_pb2.NormalizedLandmark(x=landmark.x, y=landmark.y, z=landmark.z) for landmark in detection_result.pose_landmarks[0]])
pose_proto_lm = count_landmarks(pose_landmarks_proto.landmark)
print(f"# of pose Landmarks out of boundaries: {pose_lm}")
print(f"# of normalized pose landmarks out of boundaries: {pose_norm_lm}")
print(f"# of proto pose landmarks out of boundaries: {pose_proto_lm}")
Other info / Complete Logs
# of pose Landmarks out of boundaries: [0, 19]
# of normalized pose landmarks out of boundaries: [0, 19]
# of proto pose landmarks out of boundaries: [0, 19]
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1718176661.923482 43351 task_runner.cc:85] GPU suport is not available: INTERNAL: ; RET_CHECK failure (mediapipe/gpu/gl_context_egl.cc:77) display != EGL_NO_DISPLAYeglGetDisplay() returned error 0x300c
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
/usr/local/lib/python3.11/site-packages/google/protobuf/symbol_database.py:55: UserWarning: SymbolDatabase.GetPrototype() is deprecated. Please use message_factory.GetMessageClass() instead. SymbolDatabase.GetPrototype() will be removed soon.
warnings.warn('SymbolDatabase.GetPrototype() is deprecated. Please '
Have I written custom code (as opposed to using a stock example script provided in MediaPipe)
Yes
OS Platform and Distribution
Ubuntu 22.04.4 LTS
Mobile device if the issue happens on mobile device
No response
Browser and version if the issue happens on browser
No response
Programming Language and version
Python
MediaPipe version
0.10.13
Bazel version
No response
Solution
Pose
Android Studio, NDK, SDK versions (if issue is related to building in Android environment)
No response
Xcode & Tulsi version (if issue is related to building for iOS)
No response
Describe the actual behavior
NormalizedLandmark gives some values out of boundaries. The only difference between pose_landmarks and NormalizedLandmark is that the second option contains visibility and presence scores. I tried with latest/pose_landmarker_heavy.task and 1/pose_landmarker_heavy.task, they give the same results
Describe the expected behaviour
According to the documentation, Normalized Landmark represents a point in 3D space with x, y, z coordinates. x and y are normalized to [0.0, 1.0] by the image width and height respectively. z represents the landmark depth, and the smaller the value the closer the landmark is to the camera. I understand Mediapipe tries to make up the "invisible" joints, but wouldn't be better if it simply does not include them?
Standalone code/steps you may have used to try to get what you need
Other info / Complete Logs