search

TadasBaltrusaitis / OpenFace

OpenFace – a state-of-the art tool intended for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.
Other
6.94k stars 1.85k forks source link

Iris tracking doesn't work #756

Open abdou31 opened 5 years ago

abdou31 commented 5 years ago

Hello Tadas, I really have two problems, the first that I can't test the image with OpenFace offline, it shows me a message:

Openfaceoffline.exe has stopped working ( with the last version 2.20)

The second problem is that I can't track and get iris region landmarks, I get only the landmarks of the eye region with the model trained with CNN. Note that I have a CNN using dataset annotated by OpenFace, and I have only use x and y coordinates ( 80 values for 40 points marked ) and I didn't use z coordinates for 3D in the training

Screenshots

resulttaa

Desktop (please complete the following information):

Additional context Waiting for your help, Thanks.

TadasBaltrusaitis commented 5 years ago

Just a quick question, did you use the 32 bit or 64 bit binaries?

abdou31 commented 5 years ago

64 bit binaries

TadasBaltrusaitis commented 5 years ago

Do you have the Visual C++ Redistributable installed? You would need it to run the GUI.

What do you mean by

I can't track and get iris region landmarks, I get only the landmarks of the eye region with the model trained with CNN.

There are 56 eye region landmarks that are output in addition to the 68 face ones.

abdou31 commented 5 years ago

The first problem ( not very important to me ) because I don't need to use OpenFaceofflie.exe just I need to know the problem and I know the position of landmarks drawn. The second problem : For eye region localisation and detection, I take only landmarks of eye contours and landmarks of the iris region => 40 landmarks. You know that I have used OpenFace for annotation ( FaceLandmarksImg.exe ) and I use only 2D points. When I try to detect the eye region with my CNN trained with custom dataset annotated by OpenFace tool as I told you, I found that when the iris changes her place, the landmarks did not follow the iris ( I mean there is no tracking for the iris region in real-time), it track only the eye region as you can see on the image posted in the previous comment there are only eye region points. Did you understand what I mean?

TadasBaltrusaitis commented 5 years ago

There should be 56 eye landmarks (not 40), 28 for each eye, this should include eyelids, iris, and pupil.

Do you mean that iris tracking is inaccurate? If you use OpenFace to annotate some data it is likely that you would introduce some error. The eye landmarks in OpenFace were trained on a synthetic dataset and will not always be accurate in more complex settings (or when iris is not clearly visible)