search

LSIbabnikz / eDifFIQA

This is the official repository of "eDifFIQA: Towards Efficient Face Image Quality Assessment based on Denoising Diffusion Probabilistic Models" accepted in IEEE TBIOM (Transactions on Biometrics, Behavior, and Identity Science).
14 stars 2 forks source link

How to use this model? #3

Open KylinMountain opened 5 months ago

KylinMountain commented 5 months ago

Hi, Thanks for your great work, it is very interesting. I have tested the DifFIQA, but I am not sure how to use this model?

If my face recognition model is AdaFace, should I combine the AdaFace model and DifFIQA model together and then export it a single model file like onnx? Or I can get the 512 face features from AdaFace and then input it to the DiFIQA to get score?

LSIbabnikz commented 5 months ago

Hi, This is a separate model to other FR model (such as AdaFace), it takes as input a face image (or many in a batch). So you should feed eDifFIQA face samples, which have been preprocessed similarly as with AdaFace. Specifically, the model uses an image size of 112x112, and a per channel normalization using mean of 0.5 and std of 0.5. The model will, for a given input face sample, output a quality score (single number), the higher this score the better the quality of the image. Alternatively, if you alter the configuration of the model and set return_feat=True (this can be done when initializing the model or by editing the configuration files when using the inference.py script), it will return both a quality score and a feature for the sample. The performance of eDifFIQA is close to that of AdaFace when it comes to face recognition, so you could use eDifFIQA for both feature extraction and quality prediction.

KylinMountain commented 5 months ago

Thanks for your detail answer. I will try this new eDifFIQA.

KylinMountain commented 5 months ago

@LSIbabnikz Hi, I have tested this, I have downloaded the ediffiqaL.pth. But from the inference code, it still need a base model weights like r100.pth. So if I want to export it to ONNX, I should export the model from this model model, trans = construct_full_model(conf_loc)? image

LSIbabnikz commented 5 months ago

The R100.pth file should also be available in the same folder as the ediffiqaL.pth. If you are using the official code to construct the model (construct_full_model function) than sadly you will need both weight files. The R100 weights are the pretrained weights of a CosFace (feature extraction) model obtained directly from the InsightFace repository, as this is the starting model from which we then fine-tune the eDifFIQA models.

When doing inference, this means that the weights of the backbone (feature extraction) R100 model are first loaded, then overridden by the eDifFIQA weights. This is done so that the same function construct_full_model can be called during training of the models and during inference.

As you suggested, the model you should export to ONNX is the one obtained from the construct_full_model function call.

KylinMountain commented 5 months ago

I have exported the score to the onnx. From my test on about 1000 faces captured by camera in the wild, the quality score of aligned face is more accurate and stable than not aligned.

This performance of this model is very great, thank you very much~ 👍👍👍