Closed HuangChiEn closed 2 years ago
Hi! You can control the scheduler step to be either per batch or per epoch with the interval
parameter. We default to the first, since for linear warmup, you don't waste a full epoch with a very small learning rate. If you do that, you need to adjust the other parameters accordingly.
Hi! You can control the scheduler step to be either per batch or per epoch with the
interval
parameter. We default to the first, since for linear warmup, you don't waste a full epoch with a very small learning rate. If you do that, you need to adjust the other parameters accordingly.
TKS for your reply. So, does the code snippet performs the step-level warmup instead of epoch-level in the first case (for saving the training time) ?
Yes, but this is not related to training time.
First, thanks for this amazing github repo 👍
I have a question about the implementation of learning rate scheduler. I have dig into the document of LinearWarmupCosineAnnealingLR scheduler in pytorchlightning bolt package. Most of the parameters should be the epoch-level information, such as warmup"epochs", max_"epochs". I'm wonder why the code snippet in base.py fed the step-level information to the scheduler ? (as the follows code snippet)
Any suggestion will be appreciated!!