Issue with accruacy

[jerrylouis]

Hi mndar,

I appreciate your work here, I have a few questions:

1. Could you please tell me how you validated this implementation, and what results did you obtain?
2. What models were used? I'm using this model: which loads and works fine.
3. Did you have a chance to test this against the LFW dataset (a 10 person dataset), as explained here: https://github.com/davidsandberg/facenet/wiki/Train-a-classifier-on-own-images

I have tried this using the above and the accuracy seems to be way off: see the results in the attached file. The test result is just for Ariel_Sharon! TestOutput.txt

Any pointers would be really helpful.

Thanks, Jerry

[mndar]

Hello Jerry, I haven't tested this C++ classifier with methods in David's repo but so far I have tested with 2000 users from Casia-WebFace database.

Regarding the problem you are facing:

1. Are your images 160x160 ?
2. Are you doing preprocessing on them?
3. Can you please post the "./classify ..." command you are using?

Regards Mandar Joshi

[jerrylouis]

Hello Mandar,

Thank you for your response.

I'm using the VGGFace2 model.

1. Yes, all the images are resized to 160x160.
2. I'm doing the basic preprocessing as in your preprocess_input_mat function.
3. ./facenet_classify CLASSIFY "20180402-114759.pb" "Test" "model" "labels.xml" <Classifier/Class/Labels> "MLP"

Thanks, Jerry

[mndar]

May be you are doing everything correctly, but just to make sure...

1. You have to train the SVM, KNN or MLP classifier first
2. So, your first step should be ./classify TRAIN /path/to/20180402-114759.pb /path/to/image/directory/structure model.knn labels.txt KNN

and then ./classify CLASSIFY /path/to/20180402-114759.pb /path/to/image/directory/structure model.knn labels.txt KNN

[jerrylouis]

Hi Mandar,

1. I did train first and generated the model
2. Then did the classification

I see that lot of the difference is to do with preprocessing the dataset, are you aligning the face, if so what face landmark detectors are using MTCNN or dlib. I tried this with dlib and I have an accuracy close to 60%.

See my preprocessing fn attached:

void FacenetClassifier::preprocess_input_mat (long start_index, long end_index) {
    long i;
    for (i = start_index; i < end_index; i++) {
        Mat &image = input_images[i];

        std::vector<dlib::rect_detection> rects = getFaceRect(image);

        if(rects.size() > 0)
        {
            std::vector<dlib::matrix<dlib::rgb_pixel> > desc = getAlignedFaces(image,rects[0].rect);

            dlib::matrix<dlib::rgb_pixel> spimg = desc[0];
            image = dlib::toMat(spimg);
            cvtColor(image,image, CV_RGB2BGR);
            cv::resize(image, image , cv::Size(160,160), CV_BILATERAL);

            cv::Mat temp = image.reshape(1, image.rows * 3);
            cv::Mat     mean3;
            cv::Mat     stddev3;
            cv::meanStdDev(temp, mean3, stddev3);

            double mean_pxl = mean3.at<double>(0);
            double stddev_pxl = stddev3.at<double>(0);
            cv::Mat image2;
            image.convertTo(image2, CV_64FC1);
            image = image2;
            image = image - cv::Vec3d(mean_pxl, mean_pxl, mean_pxl);
            image = image / stddev_pxl;

            cv::normalize(image, image, -1, 1,cv::NORM_MINMAX, CV_8UC1 );//This line doesnt improve the performance.

        }
    }
}

Although before I tried with the basic meanstdev as you had done and still got worse result. Do you think my approach is right in this attached code snippet?

Looks like I'm definitely missing something.

Thanks, Jerry

[mndar]

Hello, I am not able to pin point where things are going wrong for you. Perhaps I'll have to test with LFW myself. I think I have done so in the past, just can't remember for sure

This user had problems with jpg images. https://github.com/mndar/facenet_classifier/issues/4 This is a long shot but can you try png images instream of jpg?

Regards Mandar Joshi

[seungyoungshin]

Hi, Mandar,

Could you explain more detail about arguments for using TRAIN?

I prepared own my data. There are 160x160 images in each folder and I set root of image path as /path/to/image/directory/structure

├── pic │   └── shinyab │   ├── shinyab1.png │   ├── shinyab2.png │   ├── shinyab3.png │   ├── shinyab4.png │   └── shinyab5.png

I don't know how to prepare model.knn,

I type the below. /classify TRAIN ../vggface2/20180402-114759.pb ../pic model.knn labels.txt KNN

And I face some error

2018-06-22 18:12:52.163882: E tensorflow/core/common_runtime/session.cc:69] Not found: No session factory registered for the given session options: {target: "" config: } Registered factories are {}. Segmentation fault (core dumped)

Thanks

[mndar]

TRAIN takes the 128 coefficient vector and KNN (last argument) creates a KNN model in model.knn and saves the class labels in labels.txt.

About the 'No session factory error'... Have you compiled Tensorflow using the Makefile? I have come across this error when the Tensorflow library wasn't correctly built. Did you follow https://github.com/mndar/facenet_classifier/blob/master/docs/tensorflow_gentoo_armv7a.md ?

[seungyoungshin]

Hi, Mndar

Thank you for your answer. I didn't compile Tensorflow using the Makefile. I compiled Tensorflow using bazel. I followed Google Official Installation Guide.

Below is compile option I used bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package

After compiling, there is no error message and I got two shared liibary(tensorflow_cc.so, tensorflow_framework.so)

I read your compile guide insturction. I confuse because my machine is X86.

Do I have to cross compile for X86? Or this project run only on Arm series?

According to your mention, model.knn and labels.txt will be automatically created after finishin train

Thank.

[seungyoungshin]

Hi, Mandar.

I have solved my run time error. Bazel build options are changed like below.

bazel build --config=monolithic //tensorflow:libtensorflow_cc.so As I know, "--config=monolithic" option is not good for efficient.

I'm trying to do train own my images. According to your guide, 160*160 png images are used. The prediction accuracy is lower than I expect.

How many images do I need for training? Are there any rules or guidelines to improve accuracy?

Thanks.

[mndar]

How are you extracting the 160x160 face?. I've used OpenCV and Dlib for doing that.

[seungyoungshin]

I prepared manually 160x160 face images. I cropped a face and resize cropped image as 160*160. There are 8 persons. Each folder has at least two different images.

[mndar]

I've never tested with manually cropped images. Please try extracting the face with OpenCV or Dlib.

[seungyoungshin]

As you say, I have prepared train data with face detection algorithm. Actually, I used David Sandberg's python code to get train & test images. However, I got a low accuracy.

I wonder how you prepare your own data and verify. Could you share your train and test images?

I think, the training time for 6 classes is verfy short. Each classes has at least 10 images. It tooks under 1 minutes for train Is it correct?

[mndar]

Training time for 6 classes will be short. Which classifier algorithm did you use? KNN, SVM or MLP ? I'll try David Sandberg's Python code to extract images, compare them with mine and let you know. Edit: I have tested this classifier using the 20170512-110547.pb pre-trained model, not the 2018 ones.

[mndar]

I tested with the 20180402-114759 and everything seems to be working ok. I have yet to try David Sandberg's MTCNN code. I used Dlib to extract faces from images. The best results I am getting are with KNN.

[seungyoungshin]

I tested All of classifier algorithm you provide. All of results gave similar accuracy.

[y9luiz]

Hello, it's possible link the

classifier.pkl from the original repository https://github.com/davidsandberg/facenet with your implementation?