search

exadel-inc / CompreFace

Leading free and open-source face recognition system
https://exadel.com/accelerator-showcase/compreface/
Apache License 2.0
5.22k stars 718 forks source link

Accuracy is almost null for korean people's images #574

Closed Mudasar-Makandar closed 3 years ago

Mudasar-Makandar commented 3 years ago

Hello, thank you for your awesome work. CompreFace works really well when I tried working with the images of people from Western countries, Middle East and India. But when try people from South Korea, recognition accuracy is zero. It tags wrong people with 100% similarity score. What could be the issue?

pospielov commented 3 years ago

The accuracy of a model on different races depends on the number of examples of this race in the training dataset. We use pre-trained models provided by other libraries. The default build of CompreFace uses FaceNet library with a model trained on VGGFace2 dataset. I tried to find any info about race distribution but the only information I found is this quote: " Images are downloaded from Google Image Search and have large variations in pose, age, illumination, ethnicity and profession (e.g. actors, athletes, politicians)." So basically South Koreans could be just not represented well in this dataset.

What options do you have:

  1. Try InsightFace custom builds. They use for training another dataset - MS-Celeb-1M. The nationality distribution is not good, but I hope it's better: image
  2. You can find or create a dataset with the needed nationality (or nationalities) and train your own model and put it into CompreFace. Sounds not easy, but if you do the training, we will help you to add it to CompreFace
  3. Just wait until we fix it ourselves. We know about this problem and we want to fix it in CompreFace. The problem was that there was no good dataset with fair race distribution. But now there is at least one. We will work on creating a model based on it, but not sure when we release it as our resources are limited.
Mudasar-Makandar commented 3 years ago

I tried InsightFace custom builds. Mobilenet works best in my case. At least 3 images for each subject give the best recognition accuracy.

Thanks for your suggestion.

neonleo commented 1 year ago

I use arcface for my test image set - 1000 person, each person use 1 800x800 clear image for training , and each person have 10 real world image capture by ip cam for benchmark.

I wrote my own test by to test all image, about 10000 test case, and the result is very good, almost 99% is correct. I didn't try mobilenet since arcface already get the job done, maybe later I have a test on mobilenrt and post result here.