Using DlibDotNet to predict new faces from large database

[betty-crokker]

Summary of your issue

I've got tens of thousands of images with hundreds if not thousands of distinct individuals. What I would like to do is similar to what Google Picasa used to provide - have the user identify a few images, and then have some code take its best guess on yet-unidentified faces for the user to confirm. I can do this brute force (take newly-identified face, compare against every previously-identified face) but that sounds like it will get painfully slow as the number of identified faces grows. Is there a clever neural net or machine learning way to compare one new face against a large database of previously-identified faces?

I also asked this question here: https://stackoverflow.com/questions/61344611/dlibdotnet-manually-cluster-and-predict-face-matching

Thanks!

[Megadino]

@betty-crokker I would not worry about the performance of faces identification, even with thousands of individuals. I have created the software similar to Google Picasa with Dlib and have a bit of experience with the facial identification task. If to think how facial recognition works with Dlib - we have the neural network that takes an image and converts it to the vector of 128 points, and this vector is the position of a point in 128-dimensions space. The neural network is trained to put points of the same face quite close to each other, while points of different faces are far away from it, so points are clusterized in 128-dimensional space already. Then having two vectors you calculate the distance between them to understand is this the same person or not.

The distance calculation is quite fast with Dlib, it takes a couple of milliseconds on i5-3570 only. And then you can parallelize it with 10, 20, 50 threads if needed. So you can check if a face belongs to any of 500 clusters within a second in one thread.

If you look for an academic approach to vectors classification, you can look at SVM or Random Forest algorithms. Accord.NET library has an implementation for plenty of algorithms, including both above.

[takuya-takeuchi]

@betty-crokker

This task you works on is very hard. If you can use pre fileter, you can reduce computing cost.

E.G. For AFIS (Automatic Fingerprint Idenfitication System), database data has many meta data. Meta datum is gender, finger pattern, age and etc. AFIS can skip extra fingerprints which does not match these meta datum for query search.

But face recognition system could not take approach. We can not judge age and gender from face. Yes, we may use hair color and eye color. But this picture pixel should be too small.

Just as @Megadino says, this is reality. It is very simple and efficiently to use huge cluster system to improve performance.

[rajhlinux]

I also have the same issue as to OP. I am building a DVR surveillance system. It will keep track of faces detected and would like to compare every face it detects to the face cluster database.

Can you provide some algorithms which are great on doing this? Because every new face detected will be added to the cluster, so this means the cluster is growing and every new face will always needs to be compared to the expanding cluster database.

[takuya-takeuchi]

@rajhlinux I have no good ideas. But I guess the following things may help you.

• Remove old detected face from cluster database
• Introduce 1:N algorithm
  • face_recognition provide example about knn (https://github.com/ageitgey/face_recognition/blob/master/examples/face_recognition_knn.py)
  • Or kd-tree (https://www.researchgate.net/publication/285431036_KD-tree_based_clustering_algorithm_for_fast_face_recognition_on_large-scale_data)