search

google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://ai.google.dev/edge/mediapipe
Apache License 2.0
27.58k stars 5.16k forks source link

face liveliness test by blinking eye #2313

Closed ersaurabh101 closed 3 years ago

ersaurabh101 commented 3 years ago

I am looking at this demo - https://storage.googleapis.com/tfjs-models/demos/face-landmarks-detection/index.html

How can i get a trigger when eye is blinked ?

If you can provide me any direction , it will be very helpful.

ersaurabh101 commented 3 years ago

Use case will be attendance capture, Before capturing i need to ensure, its human not a picture

ersaurabh101 commented 3 years ago

I tried to use this code but it doesnt trigger any message in console, if you can help and get it to work or something similar - https://gist.github.com/kleysonr/d75494f239ad0dce561a55a624920693

I just need a valid trigger on blink.

ersaurabh101 commented 3 years ago

I am sorry for that, I tried to tell you what i am doing and what i need to achieve. I came to mediapipe from here- https://blog.tensorflow.org/2020/11/iris-landmark-tracking-in-browser-with-MediaPipe-and-TensorFlowJS.html

I just need help , want to catch when eye blinks, if possible. If not , you can close this Regards

kostyaby commented 3 years ago

Hey @ersaurabh101,

I tried to use this code but it doesnt trigger any message in console, if you can help and get it to work or something similar - https://gist.github.com/kleysonr/d75494f239ad0dce561a55a624920693

Unfortunately, I can't help you with other person's code. Please, contact that person directly to get a fix

How can i get a trigger when eye is blinked ?

That can be done using MediaPipe JS solution mentioned by @sgowroji (please, explore the links in https://github.com/google/mediapipe/issues/2313#issuecomment-883153031). To detect if someone blinked, you probably need to work with landmarks from the FACEMESH_LEFT_EYE and FACEMESH_RIGHT_EYE contours. For each eye, please pick face mesh landmarks on both vertical and horizontal edges of the eye. Then, compute the distance ratio between the vertical / horizontal edge segments - that ratio will be a good target for thresholding in a heuristic. I can't tell you specifics now as I didn't play with exact values, but there will probably be a different confidence interval of the horizontal_edge_segment / vertical_edge_segment expression value when an eye is closed and it is open - you can use that to detect whether there was a blink or not.

Sorry for the lack of specifics, hopefully this rough outline of one possible algorithm is a good starting point for your project!

google-ml-butler[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

google-ml-butler[bot] commented 3 years ago

Closing as stale. Please reopen if you'd like to work on this further.