search

deepinsight / insightface

State-of-the-art 2D and 3D Face Analysis Project
https://insightface.ai
22.83k stars 5.34k forks source link

How to increase face recognition quality on my data? #1994

Open IamSVP94 opened 2 years ago

IamSVP94 commented 2 years ago

Hi! Thanks for awesome repo! I try to use it on my data and something get wrong. Pls, help me with that

1) I make some face embeddings with "R100 | Glint360K onnx" recognition model For embedding I use cv2.dnn.blobFromImage with scalefactor=1.0 / 128.0, mean=(127.5, 127.5, 127.5), size=(112, 112), swapRB=True (face and flip face with a ~20px face paddings 1 QUESTION: should I do paddings here?)

NOT ONLY THAT PHOTO (SORRY BRUCE)!

2) I detect "new face" on the photo (RetinaFace)

3) I do 2 new face embeddings (face and flip face)

4) I try to calculate emb distance with sqeuclidean metric with all persons:

for example:

  1. new face & Bruce1 face
  2. new face & flip Bruce1 face
  3. flip new face & Bruce1 face
  4. flip new face & flip Bruce1 face
  5. new face & Bruce2 face
  6. new face & flip Bruce2 face
  7. flip new face & Bruce2 face
  8. flip new face & flip Bruce2 face

(And the same for each photo and person)

5) I get mean(all person distances) and get final distance number (for each person)

6) I labeling "new face" with "nearest person" by distance number form 5)

What could be the problem with my data? (Oh, questions... Finaly!)

1) Should I do paddings for face embedding? 2) bad jpg quality original image? 3) Should I do all this embedding manipulations or did I misunderstand something? 4) Should I align face? 5) How head turns affect quality? 6) how lighting and background affect quality? 7) Should I train the model on my data to improve the quality?

P.S. Thanks in advance! Sorry for my bad English :)

IamSVP94 commented 2 years ago

https://github.com/IamSVP94/face-detection-and-recognition/blob/56f7ee78dd2a6028ad00e62cdc611c15ba8a1dc7/src/IResNet100/utils.py#L12 - this example how I do embedding

luisfmnunes commented 2 years ago

if you used R100 | Glint360K onnx model, these models are highly sensitive to non-aligned faces, since glint360k (the training set) is a model containing only aligned faces. I made some experiments myself either and even some slight variance in the alignment resulted is performance degradation.

IamSVP94 commented 2 years ago

@luisfmnunes thanks! What about paddings? Should I use it?

luisfmnunes commented 2 years ago

@luisfmnunes thanks! What about paddings? Should I use it?

Well the same applies for padding. The images of GLint360k are supposedly just the natural results of face alignment preprocessing performed by insightface, so much likely that padding the detected/aligned faces should only result in performance degradation. But I do encourage you to test that hypothesis by yourself.