Faces getting detected in images that are upside down

[vyshak-vijayan]

Thanks a lot for your library.It is helping me immensely. I just came across an issue when dealing with images in the wrong orientation I'm trying to detect faces in images with wrong orientation.In some cases a face is detected when the image is upside down.I'm using the Hog model. For example , please see the image below https://imgur.com/a/cIv0yZE

Here the face is detected for the image that is upside down. To solve this issue , i have tried comparing the Y coordinate of the eye and lip face points to check if the image is upside down.But every time the eye coordinate(Y) seems less than the lip coordinate(Y) even if the image is upside down.

Can you please advice

[takuya-takeuchi]

@vyshak-vijayan I checked the above image by face_recognition find_facial_features_in_picture.py.

This task is difficult. Because

• face_recognition and dlib does not provide metrics for face quality. We can not decide detected face is reasonable.
  • FRDN provides detected face confidence but it could be confused. https://github.com/takuya-takeuchi/FaceRecognitionDotNet/pull/145 . This value is from https://github.com/davisking/dlib/issues/2184
  • We can upside down the above image and re-detect face information. But no way to compare 2 results.

If you can use image recognition: e.g. resnet, efficient-net or other models, you can classify face parts area. And it may return low confidence for each face part from upside down image. Especially, there are many completely different feature between human lip and hair. So deleted human lib model could reject human hair parts.

[vyshak-vijayan]

@takuya-takeuchi Thanks a lot for your response.Can't we check if the highest Y chin point is close to the bottom of the image or not inorder to identify if the image is flipped.

[takuya-takeuchi]

@vyshak-vijayan Not bad. But your idea requires that human face SHALL be above bottom of image. If a human face are close to top of image, this approach could not work.