HandLandmarker's min_hand_detection_confidence is not consistent

[twoertwein]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

Yes

OS Platform and Distribution

Mac M2

Mobile device if the issue happens on mobile device

No response

Browser and version if the issue happens on browser

No response

Programming Language and version

python

MediaPipe version

0.10.9 (latest version working on Mac - #5168)

Bazel version

No response

Solution

HandLandmarker

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

No response

Xcode & Tulsi version (if issue is related to building for iOS)

No response

Describe the actual behavior

If a hand is detected at min_hand_detection_confidence=0.71, one would expects that it also finds a hand at all lower thresholds.

Describe the expected behaviour

It is possible to get no detections at a lower confidence threshold even though a larger threshold finds a hand. I also observed the same for FaceLandmarker but I don't have a minimal reproducible example for that at the moment.

Standalone code/steps you may have used to try to get what you need

import mediapipe as mp
from mediapipe.tasks.python import BaseOptions
from mediapipe.tasks.python.vision import (
    HandLandmarker,
    HandLandmarkerOptions,
)

base_options = BaseOptions(model_asset_path="hand_landmarker.task")
for threshold in range(100):
    options = HandLandmarkerOptions(
        base_options=base_options,
        num_hands=2,
        min_hand_detection_confidence=threshold / 100,
    )
    with HandLandmarker.create_from_options(options) as model:
        image = mp.Image.create_from_file("test.png")

        if model.detect(image).hand_landmarks:
            print(f"Found a hand at {threshold/100}")
            # finds a hand for [0.01, 0.36] and for [0.51, 0.71]
            # since it detects a hand for min_hand_detection_confidence=0.71,
            # it should also detect a hand at all thresholds lower than 0.71!

Other info / Complete Logs

test.png:

[kuaashish]

Hi @twoertwein,

Could you please provide us with additional findings, such as reference videos/images or logs, demonstrating the min_hand_detection_confidence is not consistent over a specific period? This will assist us in effectively communicating the issue to the team.

Thank you!!

[twoertwein]

Thank you @kuaashish for looking into the issue!

Can you please specify what type of additional logs you are looking for? The above is a minimal reproducing example, demonstrating that a hand is (incorrectly) found when min_hand_detection_confidence is in [0.01, 0.36] or [0.51, 0.71] but not when it is in [0.37, 50].