List of problems I have run into in machine learning and how I solved them

[GarlandZhang]

Problem: Tried to improve classifier accuracy by training on new images. In doing so it started to not improve (i.e. loss and accuracy remained constant). In addition, the accuracy dropped on the previous images trained. Solution: When training on new images, must add to existing train dataset. Training on just those images alone redefines the domain of training images to just that dataset. Solution 2: I originally thought that it was because my architecture was too simple so it wasn't improving. However, I learned that in fact my architecture was too complex and the weights weren't updating easily! Now my accuracy has immensely improved Solution 3: It is important to keep your datasets for each class balanced. Otherwise you might get some biased predictions to the more frequently trained class.

[GarlandZhang]

Problem: Training on failed images that were part of test set leads to overfitting. Solution: Don't train on all failed images. Keep validation and test sets separate and only test on new images with ground truth.

[GarlandZhang]

Problem: Loss seems to converge very slowly. Solution: Omg... I messed up the corresponding annotations for each image and wrote to the wrong files. Lesson is to double check ground truths are correct before training...

[GarlandZhang]

Problem: Tensorflow versions matter. I was using the TensorFlow tutorial for basic classification of the fashion model but the accuracy was immensely affected by whether or not the Tensorflow version was 2.2 (which is what it was originally written in) vs 1.13. Solution (sort of): As it seems, its not that the accuracy is worse but which type of accuracy is being measured changes. More importantly, the accuracy on running it with actual test images still shows that it works well (~87%). So from now on don't pay attention to the accuracy given and just do your own testing.

[GarlandZhang]

Problem: After running on yolov2-tiny (coco) for over 70000 steps (batch size 10), I still have poor detection results. Solution: I changed my model yo yolov2-tiny (voc) and it works surprisingly better. One theory I have is it might be due to less classes being trained on the original model, therefore less changes need to be made to detect a single custom class.

[GarlandZhang]

Eureka! My classifier seemed to not work when using the mobilenet. I can only include, then, that it must be the data that was the problem. I suspected it might have been because my hand data had different orientations for the same gesture, so the classifier was having a hard time classifying properly. After I fixed the orientation, but still translated my hand over the video frame, (i.e. facing the camera directly), my model finally can classify the different gestures.

[GarlandZhang]

Problem: It seems that adding more classes to mobilenet, after already being trained on 3 classes, does not improve the predictions for the new classes. Is it better to just start over from scratch? Solution: While far from perfect, I found training a new mobilenet model with the added classes led to better prediction accuracy than re-training an existing model.

[GarlandZhang]

Problem: Attempted generalizing dataset to include background noise but it ended up becoming misclassifying more often. For now, I might consider requiring a white neutral background to simplify the model. Will also need to throw out all the additional data

[GarlandZhang]

Data collection: Make sure you move hand on plane to prevent hand size changing. Also make sure sleeve is not too close to hand, otherwise bounding box will include it (as noise). Also make sure enough space for hand to remove around neutral background. Also move hand farther back so its smaller to annotate...

[GarlandZhang]

Make sure that you preprocess the input the same way the library did in order to properly get correct values...this affects the corresponding predictions!!!

[GarlandZhang]

REMEMBER OPENCV IS BGR NOT RGB, THEREFORE VALUES ARE FLIPPED.