FaceVerification in caffe-windows

[anguoyang]

Hi, Mr.Wang, I found a directory named "faceverification" and also faceverification.cpp in caffe-windows, which is another project owned by you, is there any differences between that one and here?

I compiled the faceverification.cpp with caffe-windows successfully, however, I do not know how to train a model which could be used by it:

DEFINE_string(model, "", "The model definition protocol buffer text file.."); DEFINE_string(weights, "", "Optional; the pretrained weights to initialize finetuning. "

"Cannot be set simultaneously with snapshot.");

Net caffe_test_net(FLAGS_model, TEST);

I could not find any document on how to train the related model, there is a run_verification.bat and 2 prototext files, but I do not know how to use it :)

BTW, Is there any pre-trained model which I could use in this project? thanks a lot.

B.R Anguo Yang

[happynear]

The faceverification.cpp is just a code for testing. You need a trained model and then drag two images onto the run_verification.bat, then you will see the distance between the two faces (if exist).

The codes being used for training are all in this repo. Building a document is really a hard work and I do not want to do it. Please read the DeepID paper and get to know how to train a face verification model. Then you can find the codes you want here.

I was asked not to spread the trained model to the public. However, since a better model has been released in https://github.com/AlfredXiangWu/face_verification_experiment , I think there is no need to hide my model now. It was trained about one year ago, maybe I will release it in recent days.

Before I release my model, you may try Xiang Wu's model or VGG's http://www.robots.ox.ac.uk/~vgg/software/vgg_face/ .

[anguoyang]

Hi, Mr.Wang,

Thanks a lot for your quick response and the information you provided. I have looked through the source code of faceverification.cpp, there was no specific code for DeepID, is it a sign that all face models trained by caffe could be used in faceverification.cpp?

Anguo

[anguoyang]

I have downloaded LightenedCNN_B.caffemodel from Xiang Wu's project website and want to have a try, but how about the "weights" which defined in faceverification.cpp? is there a default or optimized value for this constant? thanks.

[anguoyang]

I modified the run_verification.bat as follow: FaceVerfication.exe --gpu=0 --model=CASIA_train_test.prototxt --weights=LightenedCNN_B.caffemodel pause

when I run this bat, I got these errors: I0124 09:41:13.860607 7600 layer_factory.hpp:76] Creating layer memory_data I0124 09:41:13.861608 7600 net.cpp:106] Creating Layer memory_data I0124 09:41:13.861608 7600 net.cpp:411] memory_data -> data I0124 09:41:13.862608 7600 net.cpp:411] memory_data -> label I0124 09:41:13.862608 7600 data_transformer.cpp:25] Loading mean file from: mea n.proto F0124 09:41:13.863608 7600 io.cpp:54] Check failed: fd != -1 (-1 vs. -1) File n ot found: mean.proto * Check failure stack trace: *

What is mean.proto mean? I could not find any document on it, thanks.

[happynear]

It seems that you know nothing about deep learning and caffe? The model and weights files are one-to-one correspondence. If you are using LightenedCNN_B.caffemodel, the model file should also be the one provided in Xiang Wu's repo.

There are more things about face detection and alignment you will come across in the future. The alignment method used by Xiang Wu is different from mine. The details are described in his paper.

BTW, as far as I know, there is no off-the-shelf solution for deep face recognition. If you just want a high accuracy face recognition model, I suggest you to use the online SDKs provided by Face++, Baidu etc.

[anguoyang]

Hi, Mr.Wang, Thanks a lot for your answer. The face detection/alignment is not a problem as I am also familiar with Dlib and OpenCV, however, I am a totally beginner on caffe.

I need to recognize faces in realtime videos(rtsp/rtp), so the online SDKs are not suitable for my applications, accuracy is as important as speed, and most important, I need to extract the face features/representation (Hex string) and store them into my own Database, with key-value pairs: (PersonID: representation) , this is our face DB for searching purpose in the future. When I want to identify a person, I just do these steps: 1.Decode the h.264 video into rgb24, and use dlib to detect the face with a rectangle 2.Get representation on this face and send it to the Face server/DB for searching 3,The server/DB returns top-10 PersonIDs which with higher similarity ratio compare with other PersonIDs

FaceNet is by far the most suitable solution for me, the accuracy is not bad, however, it is not as good as some online SDKs, such as FaceAll.cn(it is the most accurate online SDK during my testing), so I want to have a try on some other solutions these days, such as caffe.

BTW, I am not sure if the dataset and dlib/alignment have impact on the accuracy of FaceNet, just as you said, the web face images are really dirty, and FaceNet need 500,000 images at least for training, I am collecting other image datsets these days:)

We have developed facial related applications for more than 3 years, including video streaming server, management server and client, face detection front-ends, facial recognition server, etc. Now we need to integrate a deep-facial recognition algorithm into our ecosystem, just as you said, there is no off-the-shelf solution on it, and online SDKs are not suitable, we have to do it by ourselves .

[happynear]

OK, I can tell you something you should be attention to in order to use Xiang Wu's model.

1. Wu's alignment method is different from mine. He used the five-point landmark detection algorithm provided by CUHK, and he aligned the face image both horizontally and vertically, while in my implementation, only vertical alginment is applied.
2. Wu's face image size is 128x128, so the aligned image should also be resized to 128x128.
3. Wu's network was trained without mean file, so there is no need to find it.
4. The distance between two facial images can be calculated by caffe directly, these configurations are in CASIA_demo.prototxt, including a Normalization layer and a Euclidean layer.
5. The threshold of same / different person can be got by cross-validation on LFW or your own dataset. Since it is only one number, it is all ok to use whether 1000 samples or 10000 samples.

[anguoyang]

Hi, Mr,Wang, Thanks you very much for all the information you provided.

Is the detection algorithms or DeepID from CUHK open sourced and commercial friendly? if not, I will prefer to use DLib/OpenCV/Caffe, or FaceNet/OpenFace, as they use either BSD, Apache or Boost licenses, which are all commercial friendly.

It is really good to know that "The distance between two facial images can be calculated by caffe directly", I will try to find some related documentation before using it. BTW, is it possible for caffe be used to implement google's FaceNet? OpenFace use torch/fbnn which are in python language, I am not familiar with python, and it is not convenient for integration and distribution.

Have a nice day and thanks again for your time.

[anguoyang]

For the face detection, do you have plan to replace DLib with ccv(http://libccv.org/)? Dlib is really slow, especially on HD(e.g.1080P) images, and you have to pyramid_up before detection the face, otherwise, some of the faces(without enough pixels) will not be detected, but when I use pyramid_up on HD images, it could process no more than 1 frame within a second! even though based on OpenCV, GPU is not possible to be used in Dlib, it is much accurate than OpenCV, however, the performance is a big problem, it is really depressing.

[happynear]

If I would leave dlib one day, the only choice would be DL.