fixing an issue with resetting learning rate in gpflux and keras

[hstojic]

in keras and gpflux models we can use TF keras optimizers, instead of a callback learning rate schedule can be set as an instance of tf.keras.optimizers.schedules.LearningRateSchedule in which case our way of resetting the learning rate is not necessary and would throw an error

[khurram-ghani]

Is it worth replacing the original callback/original_lr code with a default/custom schedule derived from LearningRateSchedule, say something like this? Then we wouldn't need to detect the difference between two scheduling mechanisms.

[hstojic]

Is it worth replacing the original callback/original_lr code with a default/custom schedule derived from LearningRateSchedule, say something like this? Then we wouldn't need to detect the difference between two scheduling mechanisms.

we could easily do that but the user could still use the callback mode, we would then have to rely on them carefully reading the docs

[khurram-ghani]

we could easily do that but the user could still use the callback mode, we would then have to rely on them carefully reading the docs

If I understand correctly, you mean they would have to carefully read the keras docs to use their external callback properly, right? That is standard keras usage though. Seems to me it would be better to use keras where we can, but I guess we can't expect our users to be always familiar with keras.

[hstojic]

we could easily do that but the user could still use the callback mode, we would then have to rely on them carefully reading the docs

If I understand correctly, you mean they would have to carefully read the keras docs to use their external callback properly, right? That is standard keras usage though. Seems to me it would be better to use keras where we can, but I guess we can't expect our users to be always familiar with keras.

No, I mean they would need to read our docs carefully, to use only one type of scheduler and not through callbacks as it would result in wrong learning rates