search

deepinsight / insightface

State-of-the-art 2D and 3D Face Analysis Project
https://insightface.ai
22.17k stars 5.28k forks source link

arcface_pytorch: inference image #1610

Open Yuheng-Li opened 3 years ago

Yuheng-Li commented 3 years ago

What preprocessing do I need to do for a face image if I want to correctly get its feature by using inference.py in arcface_torch?

I need this for my other tasks, and I have bunch of face images (from FFHQ), could any one tell me what processing (land mark detection, cropping, alignment, resizing etc) do I need to do? In other words, what kind of face image is expected by the pretrained arcface model?

Yuheng-Li commented 3 years ago

I think I got it, but hope anyone can confirm or correct me if I am wrong.

It seems work like: First fire a detector such as RetinaFace to get predicted landmarks and use norm_crop() function in recognition/subcenter_arcface/common/face_align.py to crop and wrap image. And the wrapped image is a read-to-go image for the arcface model?

sainivedh commented 3 years ago

I think I got it, but hope anyone can confirm or correct me if I am wrong.

It seems work like: First fire a detector such as RetinaFace to get predicted landmarks and use norm_crop() function in recognition/subcenter_arcface/common/face_align.py to crop and wrap image. And the wrapped image is a read-to-go image for the arcface model?

Have you tested this preprocessing step ?. I'm following the arface_torch inference script for feature embeddings but the cosine similarity for similar images is around 0.32. Tested with ms1mv3-resnet50/100 models. Any suggestions on where I was going wrong !!

Thanks

Yuheng-Li commented 3 years ago

Did you align your faces?

I did not test it on unaligned faces, but I can imagine if you do that the cos score won't be too high.

I have been following the preprocessing I mentioned earlier (fire the retinaface to get 5 facial landmark and then use their norm_crop() function to align face). I tested this on some real images, and the average cos score for a same person is roughly above 0.6

sainivedh commented 3 years ago

Thanks for the reply @Yuheng-Li and yes I have not aligned my input faces. But after going through arcface_torch.eval_ijbc.py after affine transformation for the cos similarity is calculated between template features.

What is the difference between image feature emb. and template emb. Is this explained in arcface paper ?!

Thanks

Zmmccc commented 2 years ago

请问利用inference.py提取人脸图像特征时是不是需要先把人脸检测出来再输入inference中呢？我直接输入全身照相似度很低。还有就是加载的权重时下载的model_zoo中的backbone.pth吗？