Real time face recognition

[mia-petkovic]

Hello,

I am trying to understand how to use facenet for real time face recognition from the video stream. I followed the steps from the wiki page Classifier training of inception resnet v1, but I am not sure how to use the results further and if I performed all the steps correctly.

1. I ran alignment and extraction algorithm to a folder of images containing 20 images of my colleague and me, as the idea is that our faces should be recognized from the stream. Images are 182x182 px and the output is saved in training_models_182 directory.

2. I ran alignment and extraction algorithm to lsw database and generated folder lfw_mtcnnpy_160 containing aligned and cropped 160x160px faces of celebrities.

3. I ran train_softmax algorithm where data_dir is folder generated in step 1 and lfw_dir is folder generated in the step 2.

From my understanding a classifier should have been generated and further on applied to the data coming from the stream. The content of the folder are 8 files: checkpoint, meta file and ckpt.9, ckpt.12 and ckpt.15 index and data files. How should I use this files further to check whether persons in the stream are correctly classified or not?

I would really appreciate any suggestions.

[xvdehao]

Hi, maybe these files you got are used to export the embeddings of your faces,and your should use these embeddings to train a classisfy model(in the programm is a SVM classifier),and the you can use this classifier to classify your own images(you and your colleague). This is my own opinion, hope its helpful to you :)(Maybe its not correct, hoho)

[mia-petkovic]

OK if you are correct, my question would be how to use them? That's the step that I am missing what to do with this files.

[spantazi]

@mia-petkovic You can have a look at a similar project (openface). There is a real time face recognition demo app in that project, which implements pretty much what @xvdehao proposes (an SVM on top of the extracted TF embeddings/features in order to recognize faces in real time). For implementation details take a look at the websocket-server class. If you wish to use that code in conjunction with facenet, you have to substitute the part that calculates the torch features with the facenet TF equivalent (like in line 289), as also the dlib face detection and cropping with the MTCNN.

[achbogga]

It would be really nice to have a working webcam demo for facenet face recognition in real time with mtcnn. I will try to start working on it. Can someone from facenet team also help me in this regard?

Thanks Achyut

[mia-petkovic]

I am also trying to make real time webcam solution that uses mtcnn for face detection and svm for face recognition using Tensorflow. I am stuck with adjusting the code to train and test classifier with extracted embeddings. I am using this example for face detection. It seems to me that that solution does not use classification, but only Euclidean distance between embeddings to predict the class. So my idea was to add train and classify facenet function calls on top of that. The problem is for these calls I need as input folder with normalized and aligned images and I don't know how to adjust it to work with embeddings that are already extracted. Any ideas?

@spantazi Thanks for your reply, I understand your answer in theory, but find it hard to implement. I could not even run openface solution due to some encoding/decoding issues.

[AshutoshDongare]

I have created a sample of continuous Face recognition by looping through Taking photo - MTCNN - Facenet. https://github.com/AshutoshDongare/FaceNet-IOT

[phanxuanduc1996]

Hi all, I want to learn a model that can apply facenet with webcam, camera. Has anyone had code with this direction yet? If possible, please give me the reference source. Thank you.

[Rasoul20sh]

Hi @PhanXuanDuc , Maybe this helps you: https://github.com/habrman/FaceRecognition