Turn Off Hand Detection

[linjing7]

Hi, I have cropped out a hand image and use it as the input of the hand pose estimation function. However, the function sometimes can not detect the hands and thus does not perform pose estmation. Therefore, I wonder how to turn off the hand detection and treat the whole image as a hand? That is, I want to do pose estimation without box detection.

[kuaashish]

Hi @linjing7, Could you please elaborate your query with complete details and followed steps to reproduce the issue from our end. Thank you!

[linjing7]

Hi, @kuaashish, thanks for your reply. I crop out a hand image named "hand.png" and use it as input of the hand pose estimation function. Howerver, the handedness is None and thus do not perform the pose estimation.

import cv2
import mediapipe as mp
mp_drawing = mp.solutions.drawing_utils
mp_drawing_styles = mp.solutions.drawing_styles
mp_hands = mp.solutions.hands

# For static images:
IMAGE_FILES = ['hand.png']
with mp_hands.Hands(
    static_image_mode=True,
    max_num_hands=2,
    min_detection_confidence=0.1) as hands:
  for idx, file in enumerate(IMAGE_FILES):
    # Read an image, flip it around y-axis for correct handedness output (see
    # above).
    image = cv2.flip(cv2.imread(file), 1)
    # Convert the BGR image to RGB before processing.
    results = hands.process(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))

    # Print handedness and draw hand landmarks on the image.
    print('Handedness:', results.multi_handedness)
    if not results.multi_hand_landmarks:
      continue
    image_height, image_width, _ = image.shape
    annotated_image = image.copy()
    for hand_landmarks in results.multi_hand_landmarks:
      print('hand_landmarks:', hand_landmarks)
      print(
          f'Index finger tip coordinates: (',
          f'{hand_landmarks.landmark[mp_hands.HandLandmark.INDEX_FINGER_TIP].x * image_width}, '
          f'{hand_landmarks.landmark[mp_hands.HandLandmark.INDEX_FINGER_TIP].y * image_height})'
      )
      mp_drawing.draw_landmarks(
          annotated_image,
          hand_landmarks,
          mp_hands.HAND_CONNECTIONS,
          mp_drawing_styles.get_default_hand_landmarks_style(),
          mp_drawing_styles.get_default_hand_connections_style())
    cv2.imwrite(str(idx) + '.png', cv2.flip(annotated_image, 1))
    # Draw hand world landmarks.
    if not results.multi_hand_world_landmarks:
      continue
    for hand_world_landmarks in results.multi_hand_world_landmarks:
      mp_drawing.plot_landmarks(
        hand_world_landmarks, mp_hands.HAND_CONNECTIONS, azimuth=5) 

[linjing7]

In other words, how can I replace the hand bbox with my own bbox that was detected in advance? BTW, I use python.

[kuaashish]

Hi @linjing7, This might be due to the input hand image. MediaPipe models are trained to work well under "normal conditions" of good lighting and good camera quality. I was able to run the above example successfully with different similar images without error.

Could you run with several good hand images and let us know if you are still encountering the same error. Attaching the relevant gist for your reference. Thank you!

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[linjing7]

Hi, @kuaashish , thank you very much!

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No