Face too small for landmark detection

[abdou31]

Describe the bug I'm a master's degree student, and I have used OpenFace tool to annotating the dataset of 252 000 images to train my CNN for detecting eye region landmarks I have used this command

FaceLandmarkImg -wild -mutli_view 1 -aus -gaze -2Dfp -3Dfp -fdir "D:\mydata" -out_dir "D:\datas"

So after many iterations, I have got this message in the command line:

Face too small for landmark detection

I would like to know is it a warning message or an error message? Did this message mean that some images are not annotated ( there is any CSV output for some images of the folder)? If yes, how can I know verify the images that are not annotated? Did can the parametre the value of the face_confidence help me to get images detected and images not detected? Thanks.

Desktop (please complete the following information):

• OS: [e.g. iOS] Windows 10 x64
• Version [e.g. 22] 2.2.0

[TadasBaltrusaitis]

This is just a warning, and notifies that while the face detection was successful the detected face is too small for further processing.

All images will be annotated, but not all of them will have results in the CSV image. You can use face_confidence to filter more reliable versus other detections.

You can find images that are not annotated by looking for CSV files that do not have any rows corresponding to detections.

[abdou31]

Thanks for your answer, I have used a shell script to find images that haven't the csv files with same names. After annotation, I have got some invalidates pts files ( variable of eye_lmk_x conatains some negatives values ). I would like why some time Openface outputs negatives number for x and y coordinates of eye landmark?

[TadasBaltrusaitis]

Could you share some of the images where this happens?

In some cases there is a chance that negative values are not wrong (in case part of the face is not visible in the image), and the algorithm is estimating where the landmarks would be outside the image.

[abdou31]

I will share you some of the images when I found them.. ( the number of images is very big and search take more time ) I'm talking about when I extracting pts files from the folder that contains images and pts files. When I executed extracted_face_from_ibug.py, I got after some iterations an error that there is invalidation points. Note that when I execute the file I should get JSON files and their images with landmarks drawn So for pts files that contain negatives files I have seen that JSON generated from these files was empty and the image doesn't contains any landmarks drawn. I'm repeating, for now, the generating of JSON files using extracted_face_from_ibug.py and the module pts_tools.py I get this error, but the task still work after that:

[TadasBaltrusaitis]

There is no extracted_face_from_ibug.py script as part of OpenFace. So I'm not sure what script you are referring to.

Further, OpenFace outputs CSV files rather than json files.

[abdou31]

I'm so Sorry for that, You are right, this script is as part of pts tools of CNN facial landmark. (of the author Yin Guobang ). but, there is some problems on face detected. In facts, some times Openfacd detect the wrong face ( I mean not the face that have big size or region in the image ) and this make the predictions gives false predictions.

[wenliangdai]

@TadasBaltrusaitis

Hi, sorry to disturb you, but I met a similar problem so I just reply under this issue.

If I only want to extract Facial Action Units (aus), and it warns me that "Face too small for landmark detection", would this be a problem?

Also, I checked the confidence scores, some are quite high (0.88~0.975) and some are just zero.

I tried to manually lower the threshold from 0.25 to 0.2, then all this warning disappear. Is this ok?

Thank you!

[TadasBaltrusaitis]

The accuracy will be somewhat reduced, depends if the face is very small during the entire sequence or just some of the frames. The AU detection is done on bigger images so the accuracy will be reduced, especially for subtle AUs, but might be ok for larger ones (such as AU12).

[wenliangdai]

@TadasBaltrusaitis I see, thanks!