Classifier Training Error

[vahid-dan]

Hi;

I am trying to train the classifier for face detection. First, I have added a directory with my name under system/training-images containing tens of photos of me. Then clicked the "Retrain Database" button. Based on what I saw in the log messages on the screen, it seemed it could detect my face in the photos in the process of training:

=== training-images/Vahid/2017-01-25-134630_3.jpg ===
  + Writing aligned file to disk.

Aligning images took 22.2132079601 seconds.

/home/vd/home_surveillance/system/../batch-represent

But at the end, it threw this error and won't actually detect me in the application:

Representation Generation (Classification Model) took 0.551834821701 seconds.
Loading embeddings.
[2017-01-25 13:27:48,157] ERROR in app: Exception on /retrain_classifier [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "WebApp.py", line 251, in retrain_classifier
    retrained = HomeSurveillance.recogniser.trainClassifier()
  File "/home/vd/home_surveillance/system/FaceRecogniser.py", line 178, in trainClassifier
    self.train("generated-embeddings/","LinearSvm",-1)
  File "/home/vd/home_surveillance/system/FaceRecogniser.py", line 205, in train
    labels = pd.read_csv(fname, header=None).as_matrix()[:, 1]
  File "/usr/local/lib/python2.7/dist-packages/pandas/io/parsers.py", line 646, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/usr/local/lib/python2.7/dist-packages/pandas/io/parsers.py", line 389, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/usr/local/lib/python2.7/dist-packages/pandas/io/parsers.py", line 730, in __init__
    self._make_engine(self.engine)
  File "/usr/local/lib/python2.7/dist-packages/pandas/io/parsers.py", line 923, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "/usr/local/lib/python2.7/dist-packages/pandas/io/parsers.py", line 1390, in __init__
    self._reader = _parser.TextReader(src, **kwds)
  File "pandas/parser.pyx", line 538, in pandas.parser.TextReader.__cinit__ (pandas/parser.c:6171)
EmptyDataError: No columns to parse from file

Besides that, I have deleted all the predefined persons and their photos from the directories; But still it recognizes me as other ones. It seems their data is still in the database.

I really appreciate your help.

Thanks, Vahid

[BrandonJoffe]

Hi Vahid,

What is happening is that the system is attempting to read the data used to generate the classification model before the data is accessible. The labels and representations for each face image are generated by a python subprocess which in this case has actually not finished writing all the data. The data is stored in the 'system/generated-embeddings' directory in the 'labels.csv' and 'reps.csv' files.

These two files are used to train a linear SVM classifier which is stored in the 'classifier.pkl' file. The reason you are not being recognised is due to the fact that the new classifier.pkl file cannot be generated without the labels.csv and reps.csv being completed. So in a nutshell, the system should be waiting for the python subprocess to terminate before it attempts to read from the labels.csv and reps.csv files.

I actually thought I had fixed this issue it seemed to work perfectly when I tested it. Line 190 in FaceReogniser.py uses self.p.communicate() which is supposed to ensure the subprocess terminates before the program continues.

I have also tried using self.p.wait() which should, in fact, do the same thing but also did not work as expected. So I'm gonna try another combination. Could you please replace line 190 in FaceReogniser.py 'self.p.communicate()' with the two lines of code below.

self.p.stdin.close() self.p.wait()

I just tested on my system and seemed to work perfectly.

Let me know if you still receive the same error or if it comes right!

Thanks, Brandon

[vahid-dan]

Hi Brandon;

Thanks a lot for your reply. I applied the changes to system/FaceRecogniser.py as follows:

-   outs, errs = self.p.communicate() 
+   self.p.stdin.close()
+   self.p.wait()

It still throws the same error.

Vahid

[vahid-dan]

Hi Brandon;

Do you have any solution for this?

Thanks, Vahid

[vahid-dan]

Hi @gauravkaila ;

I still couldn't train the classifier myself. Could you please kindly train the classifier using my photos and send me the system/generated-embeddings/labels.csv and system/generated-embeddings/reps.csv files? I guess that's enough for the application to recognize my face. I've uploaded my photos here: https://drive.google.com/drive/folders/0B9nFNSX2ttVtbUNRZ2RFcG5FaWM?usp=sharing

That'd be a great help and I'd be more than thankful. :-)

Cheers, Vahid

[vahid-dan]

Hi @BrandonJoffe;

Could you please do me this favor?

I still couldn't train the classifier myself. Could you please kindly train the classifier using my photos and send me the system/generated-embeddings/labels.csv and system/generated-embeddings/reps.csv files? I guess that's enough for the application to recognize my face. I've uploaded my photos here: https://drive.google.com/drive/folders/0B9nFNSX2ttVtbUNRZ2RFcG5FaWM?usp=sharing

Thanks, Vahid

[BrandonJoffe]

Hi @vdaneshmand,

so sorry I haven't been able to get back to you!

Unfortunately I don't have much time on my hands to work on this project.

I have trained the classifier with your images and you can find the model and the respective files here: https://drive.google.com/open?id=0Bxl3BQa8iqkoS1Nhbnp1bXhYMXc

It should be able to recognise your face, ping me if you still having issues and I'll get back to you as soon as I can.

Cheers, Brandon

[vahid-dan]

Hi @BrandonJoffe and thanks a lot for the files. It works fine.

Cheers, Vahid