Why did my learning rate drop from the initial lr

[sjchasel]

In every batch, I execute

loss.backward()
optimizer.zero_grad()
optimizer.step()
with warmup_scheduler.dampening():
    lr_scheduler.step()

It doesn't have a warm up process.

[Tony-Y]

Does the example code work in your environment? https://github.com/Tony-Y/pytorch_warmup/blob/master/examples/emnist/main.py

[Tony-Y]

Your code does not optimize model parameters at all because optimizer.zero_grad() is called after loss.backward(). If you cannot understand this, please read the following tutorial:

https://pytorch.org/tutorials/beginner/basics/optimization_tutorial.html#optimizer

[Tony-Y]

Did you solve this issue?