This pull request addresses an issue with the step-based learning rate decay mechanism when lr_decay_ref is set to 'step'. Previously, the learning rate was decaying too rapidly due to a bug in the decay logic. This fix ensures that the learning rate decays at the correct pace as per the step-based decay configuration.
This pull request addresses an issue with the step-based learning rate decay mechanism when
lr_decay_ref
is set to 'step'. Previously, the learning rate was decaying too rapidly due to a bug in the decay logic. This fix ensures that the learning rate decays at the correct pace as per the step-based decay configuration.