Closed HuangChiEn closed 2 years ago
The result is similar but not exactly the same. With the step setting the lr is updated at every step, while with the epoch setting it is increased only in-between epochs and then kept constant for the whole epoch. We found the step setting slightly improves the performance and also makes some methods more stable.
Oh.. thanks for your explanation, the reason is a bit out of my suppose, it's great to understand the detail implementation behind the solo-learn 👍
First, thanks for this amazing github repo 👍 However, I want to ask some questions related with the closed issue #264 . Suppose we enable the warmup_cosine scheduler, the default 10 warmup-epoch is defined in the base.py.
The code snippet implements the LinearWarmupCosineAnnealingLR setup,
In the aforementioned code snippet, why don't we just setup
warmup_epochs=10
, instead we do the following modification :I can't figure out why we did the above setup with the default action is still perform the exactly 10 "epoch" warmup finally.
My guess : some situation, maybe we can setup "397 step" warmup, which is close to 10 epoch, but not "exactly" equivalent to some epoch. So, we need to manually setup 397 instead ?
Thank you for taking a look, any suggestion will be appreciated ~