search

TadasBaltrusaitis / OpenFace

OpenFace – a state-of-the art tool intended for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.
Other
6.96k stars 1.85k forks source link

Pupil/eye detection #532

Closed underscoreyu closed 6 years ago

underscoreyu commented 6 years ago

Hello there!

Sorry for my poor english and may be not so interesting questions; i'm just a schoolboy who finds your project very interesting and inspiring.

1) Is there any document that explain algorythm of eye detection? Can I for example find an existing way or may be implement code my to detect only pupil? 2) Also can i somehow find doc or something about accuracy evaluation of face detection/gaze tracking algorythms in OF.

Thank you for work. And thanks everyone, who helps to solve issues here.

TadasBaltrusaitis commented 6 years ago

Hi,

Thanks for taking an interest in OpenFace, to answer your questions:

  1. The algorithm for eye landmark detection is described in most detail in this paper - http://www.errollw.com/publications/iccv2015.pdf, specifically section 5.1. The way it is implemented, the face is detected first, then 68 facial landmarks, and only then the eye region landmarks, so you need a full image of the face for it to work. However, it is possible to first detect only the eye and initialize the eye region landmark detection, but I have not tried this. You can have a look at eye detection models from OpenCV (https://dzone.com/articles/face-and-eyes-detection-opencv) and Matlab (https://uk.mathworks.com/matlabcentral/answers/120449-how-to-detect-eyes-and-mouth-on-a-single-image-in-matlab)
  2. You can find the detailed evaluation of algorithms in the OpenFace 2.0 paper - https://www.dropbox.com/s/ki1inrjcjbwm24j/FG2018_camera_ready.pdf?dl=0

Thanks, Tadas