Closed hstojic closed 1 year ago
Is it worth replacing the original callback/original_lr
code with a default/custom schedule derived from LearningRateSchedule
, say something like this? Then we wouldn't need to detect the difference between two scheduling mechanisms.
Is it worth replacing the original callback/
original_lr
code with a default/custom schedule derived fromLearningRateSchedule
, say something like this? Then we wouldn't need to detect the difference between two scheduling mechanisms.
we could easily do that but the user could still use the callback mode, we would then have to rely on them carefully reading the docs
we could easily do that but the user could still use the callback mode, we would then have to rely on them carefully reading the docs
If I understand correctly, you mean they would have to carefully read the keras docs to use their external callback properly, right? That is standard keras usage though. Seems to me it would be better to use keras where we can, but I guess we can't expect our users to be always familiar with keras.
we could easily do that but the user could still use the callback mode, we would then have to rely on them carefully reading the docs
If I understand correctly, you mean they would have to carefully read the keras docs to use their external callback properly, right? That is standard keras usage though. Seems to me it would be better to use keras where we can, but I guess we can't expect our users to be always familiar with keras.
No, I mean they would need to read our docs carefully, to use only one type of scheduler and not through callbacks as it would result in wrong learning rates
in keras and gpflux models we can use TF keras optimizers, instead of a callback learning rate schedule can be set as an instance of
tf.keras.optimizers.schedules.LearningRateSchedule
in which case our way of resetting the learning rate is not necessary and would throw an error