Closed Nacriema closed 2 years ago
Hi thanks for the interest in the project! Indeed after a quick inspection this part is buggy, it is a feature I started to implement but I have never used it in the end. I will remove it, if you are interested in other schedulers there are now great built-in modules in pytorch (see https://pytorch.org/docs/stable/optim.html) Best, Tom
Hi, thank you for your response. Your code is very clean, I learned a lot from your coding style. Thank you very much !
Hi, from this discussion thread: https://discuss.pytorch.org/t/solved-learning-rate-decay/6825/4, I have been implemented the PolynomialLR scheduler, but I'm not sure if it is correct. Could you please check it, thank you in advance:
class PolynomialLR(_LRScheduler):
def __init__(self, optimizer, max_iter, decay_iter=1, power=0.9, last_epoch=-1):
self.decay_iter = decay_iter
self.max_iter = max_iter
self.power = power
super(PolynomialLR, self).__init__(optimizer, last_epoch)
def get_lr(self):
if self.last_epoch % self.decay_iter == 0 or self.last_epoch % self.max_iter == 0:
factor = (1 - self.last_epoch / float(self.max_iter)) ** self.power
return [base_lr * factor for base_lr in self.base_lrs]
return [group['lr'] for group in self.optimizer.param_groups]
def __str__(self):
params = [
'optimizer: {}'.format(self.optimizer.__class__.__name__),
'decay_iter: {}'.format(self.decay_iter),
'max_iter: {}'.format(self.max_iter),
'gamma: {}'.format(self.gamma),
]
return '{}({})'.format(self.__class__.__name__, ', '.join(params))
Mmh from a quick look it looks good to me, to be sure you should try on a basic example and print the LR when it changes during training
Hi, this is the learning rate scheme I've got for testing:
max_iter=100, decay_iter=1, power=1.0
max_iter=100, decay_iter=5, power=1.0
max_iter=100, decay_iter=1, power=2.0
max_iter=100, decay_iter=5, power=2.0
Thank you very much for sharing the code with us. Recently, I inspect and test all the code that you provided. I notice that your custom PolynomialLR inside the schedulers packet returns the result the same as the ConstantLR does. Please let me know if it is a bug. Thank you in advance.