facerecogniton fails

[mlanandnair]

hi justdude, I have tried using your face recognition example1 and worked on face recognition using images from my mobile. i trained facerecognizer with three images for each person and gave a persons image in order to predict , but it fails most of the time predicting wrong person . can u help me out??

[justadudewhohacks]

Hi, depending on the similarity of the faces you might need more training data to get a better prediction result. In the example I just used 3 per person, but in general I would recommend using more training data.

[mlanandnair]

I have already used three images for training per person. Is there any constraints in the input image for prediction, Because when i resized the image prediction went wrong.

[justadudewhohacks]

The only constraint is, that all the images you put into the face recognizer are equally sized.

In the examples I first detect the faces, which gives me a rectangular region of each image. I resize the detection results to 80x80 images, which should kind of depend on the size of the faces in your input images.

You can try to increase the size, but consider that upscaling detection results (depends on the size of the rectangle around the face) might result in noisy input.

[mlanandnair]

I didn't changed the trainImgs size , it's still in 80*80. I have tried using a HDR Image and an image with low scale factor for prediction , when image size is low the prediction is kind of accurate . But when image size is higher like HDR image prediction fails . Sometimes it fails in recognizing person and at somepoint of time it predicts wrong person. So is there any solution to fix it . Also i have tried this

const lbph = new cv.LBPHFaceRecognizer(); lbph.train(trainImgs, labels); cv.imshowWait('test', trainImgs); const twoFacesImg = cv.imread(path.resolve(basePath, 'daryl-rick.jpg')).resize(500, 500);

when i resized this prediction image prediction went wrong. Am I doing any thing wrong here?

[justadudewhohacks]

A possible problem could be that the faces are not detected correctly. Can you imshow each face detection result of the training and test images to verify, that the faces are detected properly?

Also I would NOT recommend to do this:

const twoFacesImg = cv.imread(path.resolve(basePath, 'daryl-rick.jpg')).resize(500, 500);

resize(500, 500) will pretty much distort your input images, as it does not keep the images aspect ratio. If you want to rescale the images, while preserving the aspect ratio, you can use mat.rescale(factor) or mat.resizeToMax(500).

The reason we can apply resize(80, 80) to the face detection results (not the training images), is because the face rectangles are always width = height.

[mlanandnair]

Yes I was able to im show training imgs may be due to resize , thank you for the information dude. I will check it

[mlanandnair]

hi dude, i have tried with mat.resizeToMax(500) and was able rescale and got a bit of progress when input image is big, but prediction fails at somepoint when input is changed. I have a scenario like this, namemappings is only one name , training imgs is 3 images of a person . Then i used prediction with the same person image and image of another person. For the same person lbph confidence value is 84.72297024899063 and for another persons image it is 104.4310128893347, One more thing is in certain images with proper faceimage classifier is not detecting face . why it happens so?

[justadudewhohacks]

Do you mean detectMultiScale in general does not detect the faces sometimes, or the example code?

Note that in the example code I simply return the first face rectangle for the training images, since I assume there is only a single face in those images anyways.

There are also some parameters, which you can adjust for detectMultiScale, maybe adjusting those can help also. You should also inspect the returned detection result. It returns an objects array (the rectangles) and an numDetections array, which can be used as a metric to identify the best detection result (you can also have a look at the face detection examples)

The method signature for detectMultiScale is: { objects: [Rect], numDetections: [Int] } : cc.detectMultiScale(Mat img, Number scaleFactor = 1.1, Int minNeighbors = 3, Uint flags = 0, Size minSize = Size(), Size maxSize = Size())

[mlanandnair]

yup , "detectMultiscale in general doesnot detect the faces sometimes". got output as { objects: [], numDetections: [] } . can you explain the relevance of confidence value.

[justadudewhohacks]

Actually I am not sure what tge exact meaning of the Confidence values are. Maybe the guys at opencv forum can help you with that question.

Also I think the face recognizers will give you the best match of an input face to the faces it has been trained on. I am not sure if you can actually detect, whether an "unknown" face is actually unkown. One would then probably have to figure this out via the Confidence values somehow, but again I amnot sure about the exact meaning of the confidence values.

Regarding the issue, that sometimes faces are not detected: The parameter you pass to the cascade classifier constructor is the path to a model xml file. In case of the example this is a model trained on frontal face Images, maybe some of your example images are not necessarely frontal faces?

[mlanandnair]

can you explain frontal faces, is there any another than models than HAAR_FRONTALFACE_ALT2 in your libraries.

[justadudewhohacks]

You could also try HAAR_FRONTALFACE_ALT, HAAR_FRONTALFACE_ALT2, HAAR_FRONTALFACE_ALT_TREE, HAAR_FRONTALFACE_DEFAULT, HAAR_PROFILEFACE. Not sure if these will give you any better results though.

I exported all the haar cascades available in the opencv repo (https://github.com/opencv/opencv/tree/master/data/haarcascades) as follows:

HAAR_EYE
HAAR_EYE_TREE_EYEGLASSES
HAAR_FRONTALCATFACE
HAAR_FRONTALCATFACE_EXTENDED
HAAR_FRONTALFACE_ALT
HAAR_FRONTALFACE_ALT2
HAAR_FRONTALFACE_ALT_TREE
HAAR_FRONTALFACE_DEFAULT
HAAR_FULLBODY
HAAR_LEFTEYE_2SPLITS
HAAR_LICENCE_PLATE_RUS_16STAGES
HAAR_LOWERBODY
HAAR_PROFILEFACE
HAAR_RIGHTEYE_2SPLITS
HAAR_RUSSIAN_PLATE_NUMBER
HAAR_SMILE
HAAR_UPPERBODY

Maybe you can also find custom models on github or the web, which work better for you.

You may also check out this post for finetuning detectMultiScale parameters: https://stackoverflow.com/questions/20801015/recommended-values-for-opencv-detectmultiscale-parameters

[mlanandnair]

In your examples you are using group images for face prediction and using multiple person's for training purpose. Is it advisable to use only one person's images(minimum 3 images) for training purpose, and only one name in the name mapping array ?. I am working on a project with this scenario to predict a person. but when using some other person's image LBPHpredector returns the same name(whatever name when using same person's image). and What could be the possible output when i use a different person's images for predection which is not present in the training set ?

[justadudewhohacks]

I think you can not train a face recognizer with only one person. It should give you an error if the all the values in the labels array are equal. As the face recognizers always return the best match it will always return the same name of course.

In your case what I would try out is to train the recognizer with face images of the person (label 1) and face images of other people (label 2), such that you end up with two labels (true, false).

But I would highly suggest you to take advice from the opencv community by posting a question about face recognition in the opencv forum or on stackoverflow. They can probably give you more sophisticated advice then I am able to give to you.

[mlanandnair]

Thank's for the information dude.

[justadudewhohacks]

If you still have issues with the face detection part not returning a result for some images, I have added face detection example using a pretrained neural net from the opencv samples. I found this to provide a bit better results in case the face is not perfectly frontal.

[mlanandnair]

Thanks dude, i will check that

[justadudewhohacks]

Not sure if you are still looking for something, but I recently wanted to build a face recognition app as well and found the OpenCV Face Recognizers to not perform accurate enough, because I had faces in different poses, which are not frontal.

I am currently working on an npm package called face-recognition.js, which provides way better face recognition results using deep learning. Under the hood it uses dlib: https://github.com/justadudewhohacks/face-recognition.js

I just started this project, but my goal is to soon make this compatible with opencv4nodejs.

[justadudewhohacks]

This package can now be used with opencv4nodejs. I will close this issue here.