google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://ai.google.dev/edge/mediapipe
Apache License 2.0
27.76k stars 5.18k forks source link

Issue/Bug with Face Landmark Detection #4698

Open Shubhambindal2017 opened 1 year ago

Shubhambindal2017 commented 1 year ago

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

No

OS Platform and Distribution

Ubuntu

Mobile device if the issue happens on mobile device

No response

Browser and version if the issue happens on browser

No response

Programming Language and version

Python

MediaPipe version

0.10.3

Bazel version

No response

Solution

FaceMesh

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

No response

Xcode & Tulsi version (if issue is related to building for iOS)

No response

Describe the actual behavior

If at num_faces=2, returned/detected number of faces returned by FaceMesh is 1. Then at num_faces=1 - it should also detect that person i.e returned number of faces should be 1, right? But keeping all other parameters same but setting num_faces=1 - it is returning no face.

Describe the expected behaviour

1- One bug/issue is the one explained above.
2- Also can you also explain what is min_face_presence_confidence? I know what min_face_detection_confidence is, but not sure about 'min_face_presence_confidence' and how it is used. In output - NormalizedLandmark object also has 'presence' attribute - is it related to it?
3- Also in the image there are 4 faces, but even keeping min_face_detection_confidence and min_face_presence_confidence - to 0.02, it only detects single face (even when num_face>=2). So it seems the model is quite bad, isn't it? [With default thresholds it doesn't detect any face - out of four]

Standalone code/steps you may have used to try to get what you need

Colab - https://colab.research.google.com/drive/1niKeeZ5V0t5lcb2s1vPrWVlJRvlZy1eG?usp=sharing

It has run on the same image for different below setting:
a) min_face_detection_confidence=default, min_face_presence_confidence=default, num_faces=1
b) min_face_detection_confidence=0.02, min_face_presence_confidence=0.02, num_faces=1
c) min_face_detection_confidence=default, min_face_presence_confidence=default, num_faces=2
d) min_face_detection_confidence=0.02, min_face_presence_confidence=0.02, num_faces=2

Other info / Complete Logs

No response

kuaashish commented 1 year ago

Hello @gkarpiak, @ivan-grishchenko,

Could you please look into this. Thank you

Shubhambindal2017 commented 1 year ago

Hi @ivan-grishchenko @gkarpiak @kuaashish, can you please help on this?

Shubhambindal2017 commented 1 year ago

Hi, @ivan-grishchenko @gkarpiak @kuaashish, the issue still seems to be in the latest versions. Can refer to the shared colab