Check if you model work on the training data set. A good idea can be to verify that
you can overfit to a smaller portion of the training data. If not, your model is not
working, you might have an error somewhere, or your data might not be trainable.
∙ Run a few epochs (iterations through all traning samples) using varying learning rate
and inspect plots of the loss function and the accuracy.
∙ Check your amount of data compared to the model complexity: should you consider
traning from scratch or using pretrained model/transfer learning (next week).
∙ Check preprocessing/data standardization.
∙ Check the quality of your labels (sometimes needed for new data sets).
∙ Hyperparameter search is something you start after a good set of experiments, when
you have confidence that your model is working.
∙ Watch out for NaN! Error? Too high learning rate?
∙ Use regularization/dropout
Check if you model work on the training data set. A good idea can be to verify that you can overfit to a smaller portion of the training data. If not, your model is not working, you might have an error somewhere, or your data might not be trainable. ∙ Run a few epochs (iterations through all traning samples) using varying learning rate and inspect plots of the loss function and the accuracy. ∙ Check your amount of data compared to the model complexity: should you consider traning from scratch or using pretrained model/transfer learning (next week). ∙ Check preprocessing/data standardization. ∙ Check the quality of your labels (sometimes needed for new data sets). ∙ Hyperparameter search is something you start after a good set of experiments, when you have confidence that your model is working. ∙ Watch out for NaN! Error? Too high learning rate? ∙ Use regularization/dropout