This project consist of implementation of FaceNet and focus is not on the training but on the application side and deployment side It contain scripts for Face detection and Face recognition which can be directly used in other apps to addded face recognition to the project.
FaceNet is the current state of the art method to face Recognition/verification with accuracy of 99.63 on LFW dataset and 95.12 on Youtube Faces Dataset. It was first published in paper FaceNet: A Unified Embedding for Face Recognition and Clustering by amazing Florian Schroff, Dmitry kalenichenko, james Philbin from google research team.
Rather than classifying images directly from a softmax classifier or with siamese network. FaceNet goes to a completely different route, A new loss called as triplet loss is introduced in this paper. This loss function is as follow
where,
This loss work on euclidian distance between the three images, To train a network yourself you will need three images per example (i.e. one anchor image one positive and one negative) this loss function tries to reduce the euclidian distance between anchor and positive image and tries to increase the the euclidian distance between anchor image and negative image.
learn more about triplet loss here
I have taken the pretrained InceptionResNet model from nyoki-mtl github repo This repo contained model weights in older version of keras, so when I tried to load model weigths in newer version of tensorflow I got 'bad marshal error' Then I realized that since keras was integrated in tensorflow as backend the saving formate is changed and those all keras model do not load in newer version. I tried bunch of thing to load them in latest version, Finally I found the soultion I did following things
You can export the model in single keras .h5 format. I did excatly that to use the model as pre trained model for transfer learning in another project. It had differnt version of Tensorflow it still gave 'Bad marshal' error. So I recommend using the layer by layer weights they are independent of the tensorflow version :smile:. To load the weights look 'load_model' method in Inference/FaceRecognizer.py file
folders discription
rest folders i.e. Inference and Training contains code, ignore them
Follow the steps as follow
pip install -r requirements.txt
Your project should have a DataBase and Model folder in root directory. just copy folder Inference and utils.py in you ptojects root folder. then import detector class from utils that's it. Open app.py file to see how to use detector class. Documentation about arguments and output is given in utils.py as comments.
F.R.I.E.N.D.S is one of the best sitCom ever, So let's see out Recognizer can recognize the cast correctly.
Original image | Resulting image |
---|---|
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
images are taken only for educational purpose In some cases the model guessed Rachel wrong in image 5 and image 6 In image 5 the image is of older Jennfier aniston that is why our code misclassified it although threshold is set to 1 in the code that is anyone greater than ruclidian distance greater than 1.0 is tagged as UNKNOWN you can also see that the distance is 1.03 very close In image 6 the the Face is tilted to side a little hence the model got confused there
Upcoming Features:- 1) an Flask based app to use the code as a web app (deployed version) 2) Docker image of flask app ready to deploy
Features the Flask app will have 1) selecting image fro mdirectory 2) Detecting faces in video or realtime feed from web cam or some online stream source
Sometime in future (When the resources will be available I do not have rpi or arduino :sob:):- 1) EfficientNet based small but accurate model 2) Tiny model which can be used in Microcontroller 3) Integration to use the app with Arduino and raspeberry pi