lr_scheduler.step() is being called before optimizer.step()

[hadasah]

When using train_model.py, I get this error:

/private/home/margaretli/.conda/envs/latest/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:82: UserWarning: Detected call of lr_scheduler.step() before optimizer.step(). In PyTorch 1.1.0 and later, you should call them in the opposite order: optimizer.step() before lr_scheduler.step(). Failure to do this will result in PyTorch skipping the first value of the learning rate schedule.See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate

[domrigoglioso]

@hadasah Would you be able to provide a specific command that causes the warning to show?

[stephenroller]

pretty much if you provide --lr-scheduler invsqrt to any of the models

[hadasah]

^^ what stephen said. But if you need a specific invocation to get started:

python examples/train_model.py -t convai2 -m transformer/ranker -mf /tmp/tr0 --lr_scheduler invsqrt --warmup_updates 50