Facial features not detected when .pose generated from video

[MuhammadAasharibNawshad]

Hi, i am using the provided script for generating .pose from video. But facial feature are not detected. instead brown circle is appearing.

Also note that on top left there is some unwanted animation which i guess is same pose but with some translation.

Can you please tell how to fix that?

https://github.com/sign-language-processing/pose/assets/22044917/0030aed2-cd30-4739-90b8-4631dbb6d3a0

[AmitMY]

1. The facial features are detected, it is just that this is drawn in a low resolution, and as such all of the lines intersect. You could visualize the pose with thickness=1 or increase the resolution or both

from pose_format import Pose
from pose_format.pose_visualizer import PoseVisualizer

with open("example.pose", "rb") as f:
    pose = Pose.read(f.read())

pose.body.data *= 2 # scale pose values
pose.header.dimensions.width *= 2 # scale video dimension
pose.header.dimensions.height *= 2 # scale video dimension

v = PoseVisualizer(pose, thickness=1)

v.save_video("example.mp4", v.draw())

2. The thing at the top left is a 3D body world landmark representation. You can always remove it by selecting all the other components

   pose = pose.get_components(["POSE_LANDMARKS", "FACE_LANDMARKS", "LEFT_HAND_LANDMARKS", "RIGHT_HAND_LANDMARKS"])

[MuhammadAasharibNawshad]

Thankyou for clarification. Point 1 worked fine. But for the point 2, can you please tell exactly file & method in which change is to be done. Currently i am assuming change in src/python/pose_format/utils/holistic.py formatted_holistic_pose method. in which return statement is return pose.get_components(["POSE_LANDMARKS", "FACE_LANDMARKS", "LEFT_HAND_LANDMARKS", "RIGHT_HAND_LANDMARKS"], {"FACE_LANDMARKS": FACEMESH_CONTOURS_POINTS})

Actually what i need to fix for point 2 is black circled thing removal from .pose / video

[AmitMY]

The second step is before you are visualizing to select what you want, so it goes right after loading the file, not changing directly in the repository.

with open("example.pose", "rb") as f:
    pose = Pose.read(f.read())

# Remove unwanted stuff
pose = pose.get_components(["POSE_LANDMARKS", "FACE_LANDMARKS", "LEFT_HAND_LANDMARKS", "RIGHT_HAND_LANDMARKS"])
# or
from pose_format.utils.generic import reduce_holistic
pose = reduce_holistic(pose)

# Visualize
pose.body.data *= 2 # scale pose values
pose.header.dimensions.width *= 2 # scale video dimension
pose.header.dimensions.height *= 2 # scale video dimension

v = PoseVisualizer(pose, thickness=1)

v.save_video("example.mp4", v.draw())

[MuhammadAasharibNawshad]

ok thankyou so much! it worked!