statmike
commented
2 years ago Great catch! You are correct, the current values of the hyperparameters need to be passed to the optimizer definition in this example. It turns out it was running without error because it was just using the default values for learning_rate and momentum. This shows up in the results as the exact same value for loss regardless of the turning run. I am making the correction and will push the update after I test it today. Thank You!!
The examples in 05h and 05i don't have the mistake and take advantage of TensorBoard HPARAMS parallel coordinates view which would have likely helped me diagnose/notice this issue.
In the cell where the {DIR}/train.py is being written you have declared the arguments --lr (learning_rate) and --m (momentum) in the parser.
However, those variables are not being used in the optimizer (SGD).
I think the correct definition of the optimizer will be:
opt = tf.keras.optimizers.SGD(learning_rate = args.learning_rate, momentum = args.momentum)
Am I correct? Or how can the hyperparameters be connected with the training of the model??
Thank you for your work.