Confidence score in the Iris tracking solution

[vovaani]

Hi,

I'm using the Iris tracking solution in Python (after applying the patch described here). I'm interested in identifying frames in which the Iris is not visible because of obstruction by a glasses frame / a hand or something else.

I thought to use a confidence score for this purpose, however I've noticed that unlike face mesh, the Iris solution does not export any score of confidence (explored the iris_landmark.tflite file and it does not seem to have confidence tensors exported).

Is it possible to export a confidence score in the Iris solution? Alternatively, is there another good solution in my case?

Thank you! Vova

[vovaani]

Great, thanks for the super quick answer! I'll try that.

Just to make sure - I see the face landmark detection output stream which contains the detection message that AFAIU covers all the landmarks, except the ones added exclusively by Iris.

Am I missing something? If not, is there a detection message for the Iris landmarks as well?

Thank you!

[vovaani]

So I extracted the face detections and here are a few findings:

1. There is a global score for the entire face detection (as opposed to a score per landmark as I thought previously).
2. The score seems to reflect major obstructions of the face. So for example if I put a hand in front of my eyes the score goes down from ~0.9 in this image -

to ~0.8 in this image -

3. The score does not seem to reflect small obstructions, like glasses that obstruct the eye with the frame. The score remains 0.9 in the following image -

Given these findings it seems that I can probably solve part of my problem using the score from face detections, but not all of it. Is there something else I should consider?

Thank you!

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[vovaani]

Thank you! Will check and update here.

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No

[vovani]

Following up - we were successful in using 'refine_landmarks' to extract the iris landmarks directly from face_mesh. This is very convinient, so huge thank you for that!!

Howevee, the issue described here still stands - the confidence score is a global score, and we did not find a reliable way to use it in order to understand if someone's eyes are visible or not.

[ayushgdev]

Hello @vovani Are you still looking for a resolution on this issue?

[vovaani]

Hello @ayushgdev,

Yes, it would be nice to have a resolution for this issue!

Thank you,

[arianatri]

soooo? nothing yet?

[kuaashish]

Hello @vovani, We are upgrading the MediaPipe Legacy Solutions to new MediaPipe solutions However, the libraries, documentation, and source code for all the MediaPipe Legacy Solutions will continue to be available in our GitHub repository and through library distribution services, such as Maven and NPM.

You can continue to use those legacy solutions in your applications if you choose. Though, we would request you to check new MediaPipe solutions which can help you more easily build and customize ML solutions for your applications. These new solutions will provide a superset of capabilities available in the legacy solutions. Thank you

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed due to lack of activity after being marked stale for past 7 days.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No