Cause for bad classification on live and video demo

adriandavidauer / VVAD

Repo about VVAD

GNU General Public License v3.0

3 stars 1 forks source link

Cause for bad classification on live and video demo #22

Open adriandavidauer opened 3 years ago

adriandavidauer commented 3 years ago

The classifications are obviously wrong on the video and live demo. Causes could be the following:

~~WARNING:tensorflow:Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.~~
FPS are not the same
twisted dimensions in the array may be a problem

adriandavidauer commented 3 years ago

it will predict right. The "optimizer" is only necessary for "training", and it's state will only help select the proper learning rate and weight updates. Even with a fresh new optimizer, it will be possible to train if you manually adjust its properties to reasonable amounts. A fresh new optimizer may have a hard time at the beginning of the training, but after a few epochs it will very probably find its way.

source: https://stackoverflow.com/questions/49195189/error-loading-the-saved-optimizer-keras-python-raspberry

adriandavidauer commented 3 years ago

if the dimensions are somehow wrong, it should look weird in the visualization of a sample or at least different to waht we know from the dataset

adriandavidauer commented 3 years ago

Tested on the pretrained faceImage model on the testset and it performs poor! -> something must be wrong with the model!!! All the other models perform as excpeted!

Accuracy for /home/alubitz/anaconda3/envs/vvadlrs3/lib/python3.7/site-packages/vvadlrs3/bestFaceEndToEnd.h5 is 0.5
MAE for /home/alubitz/anaconda3/envs/vvadlrs3/lib/python3.7/site-packages/vvadlrs3/bestFaceEndToEnd.h5 is 0.5019764296190415 with std 0.4942703556715032

adriandavidauer commented 3 years ago

FaceImages are the only ones that are being resized. Maybe something goes wrong there... The model should be fine. I still have some backup models with 94% validation accuracy and there it is the same problem...

Next I should compare how samples for training are resized and how they are resized for testing

Both just use FeaturizedSample.getData with the specific imageSize and numSteps...
- What is it that I'm not seeing?

adriandavidauer commented 3 years ago

Tests on a random video of me are poor. The pretrained models for the different featureTypes produce very different results. And all of them seem to be rather bad. Have to label a video myself for testing it a little better but first impression is bad!