FaceMesh versus Holistic in Python

[zeina-abuaisheh]

Hello,

I tried to use FaceMesh in Python, it works perfectly when the people are less than 1m away from the camera, however, when they are further, it doesn't output a mesh for them.

I did the same experiment with Holistic and tested holistic on the same video as FaceMesh, and it outputs meshes for all of the people even when they are further.

Do you have an idea why this works on holistic but not on facemesh?

Thanks

[sureshdagooglecom]

Hi @zeina-abuaisheh , can you provide code changes to investigate further on this issue.

[zeina-abuaisheh]

this is the code used in the holistic model:

mp_drawing = mp.solutions.drawing_utils
mp_drawing_styles = mp.solutions.drawing_styles
mp_holistic = mp.solutions.holistic

with mp_holistic.Holistic(
    min_detection_confidence=0.5,
    min_tracking_confidence=0.5) as holistic:
  while cap.isOpened():
    success, image = cap.read()
    if not success:
      print("Ignoring empty camera frame.")
      # If loading a video, use 'break' instead of 'continue'.
      continue

    # To improve performance, optionally mark the image as not writeable to
    # pass by reference.
    image.flags.writeable = False
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    results = holistic.process(image)
    # Draw landmark annotation on the image.
    image.flags.writeable = True
    image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
    mp_drawing.draw_landmarks(
        image,
        results.face_landmarks,
        mp_holistic.FACEMESH_CONTOURS,
        connection_drawing_spec=mp_drawing_styles
        .get_default_face_mesh_contours_style())
    mp_drawing.draw_landmarks(
        image,
        results.pose_landmarks,
        mp_holistic.POSE_CONNECTIONS,
        landmark_drawing_spec=mp_drawing_styles
        .get_default_pose_landmarks_style())

    mp_drawing.draw_landmarks(
        image,
        results.left_hand_landmarks)

    mp_drawing.draw_landmarks(
        image,
        results.right_hand_landmarks)

    # Flip the image horizontally for a selfie-view display.
    cv2.imshow('MediaPipe Holistic', cv2.flip(image, 1))
    video_writer.write(image.astype(np.uint8))

    if cv2.waitKey(5) & 0xFF == 27:
      break
cap.release()

And this is the code used in the facemesh model:

mp_drawing = mp.solutions.drawing_utils
mp_drawing_styles = mp.solutions.drawing_styles
mp_face_mesh = mp.solutions.face_mesh

mp_face_detection = mp.solutions.face_detection
mp_drawing = mp.solutions.drawing_utils

# For webcam input:
drawing_spec = mp_drawing.DrawingSpec(thickness=1, circle_radius=1)
with mp_face_mesh.FaceMesh(
    max_num_faces=3,
    refine_landmarks=True,
    min_detection_confidence=0.5,
    min_tracking_confidence=0.2) as face_mesh:
  while cap.isOpened():
    success, image = cap.read()
    if not success:
      print("Ignoring empty camera frame.")
      # If loading a video, use 'break' instead of 'continue'.
      continue

    # To improve performance, optionally mark the image as not writeable to
    # pass by reference.
    image.flags.writeable = False
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    results = face_mesh.process(image)

    # Draw the face mesh annotations on the image.
    image.flags.writeable = True
    image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
    if results.multi_face_landmarks:
      for face_landmarks in results.multi_face_landmarks:
        mp_drawing.draw_landmarks(
            image=image,
            landmark_list=face_landmarks,
            connections=mp_face_mesh.FACEMESH_TESSELATION,
            landmark_drawing_spec=None,
            connection_drawing_spec=mp_drawing_styles
            .get_default_face_mesh_tesselation_style())
        mp_drawing.draw_landmarks(
            image=image,
            landmark_list=face_landmarks,
            connections=mp_face_mesh.FACEMESH_CONTOURS,
            landmark_drawing_spec=None,
            connection_drawing_spec=mp_drawing_styles
            .get_default_face_mesh_contours_style())
        mp_drawing.draw_landmarks(
            image=image,
            landmark_list=face_landmarks,
            connections=mp_face_mesh.FACEMESH_IRISES,
            landmark_drawing_spec=None,
            connection_drawing_spec=mp_drawing_styles
            .get_default_face_mesh_iris_connections_style())
    # Flip the image horizontally for a selfie-view display.
    cv2.imshow('MediaPipe Face Mesh', cv2.flip(image, 1))
    video_writer.write(image.astype(np.uint8))

    if cv2.waitKey(5) & 0xFF == 27:
      break
cap.release()

[sureshdagooglecom]

Hi @zeina-abuaisheh ,can you please decrease min_detection_confidence value because it is detecting for 3 faces and can you please share recorded video to investigate further.

[zeina-abuaisheh]

Thanks a lot for your reply I already changed the number of faces to 1, but it doesn't change anything rather than detecting one person each time.

Here is the video I am testing on: https://drive.google.com/file/d/1sZ6lkkavQTgPhSeM5_lL7wMxzedbPPKt/view?usp=sharing

Thanks

[sureshdagooglecom]

@zeina-abuaisheh , could you check the code differences for holistic , you have not specified this in your code ,kindly check once.

[zeina-abuaisheh]

I didn't get what you mean? My problem is with the facemesh code, not the holistic one. The holistic one manages to detect the faces and generate the landmarks even when people are far away However, I am unable to get the same results when I only use facemesh

[sureshdagooglecom]

Hi @zeina-abuaisheh ,could you check this once to detect faces for different lengths.

[zeina-abuaisheh]

Thanks a lot, How to use this configuration? https://github.com/google/mediapipe/blob/master/mediapipe/modules/face_detection/face_detection_full_range_gpu.pbtxt

Which part of the python code should I change so that the above code should be taken into account?

Thanks

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler[bot]]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No

[sureshdagooglecom]

@zeina-abuaisheh , Need to use right Subgraph for different distances.

[zeina-abuaisheh]

Can you let me know how can I change the subgraph? Thanks

[gavspav]

If you look in the face_mesh.py file you can see that the face detect model is hardcoded into super().init()

I tried to change it to long range but it dodn't work.

If anyone works it out, please let me know!

[sureshdagooglecom]

Hi @zeina-abuaisheh , Building MediaPipe Android apps is still not possible on native Windows. Please do this in WSL instead and see the WSL setup instruction in the next section. Please follow the mentioned steps in this on configuring the subgraphs.

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler[bot]]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No