Closed huixiancheng closed 3 years ago
Hi, lr_scheduler is the method to set up lr decay in training, such as multi-step decay and cosine decay. We use cosine decay in this work.
I know it's cosine_schedule with warm up.
But the ques is about update the ls after a epoch or batch(iter), there are two paras: epoch/num_updates and metric.
Since I don't use timm before,I cann't understand the meaning of Para:metric
In timm it don't have special meaning too. As the code show.
You need update ls in each epoch to tell it current epoch, which is the ls usage in torch.
Thanks for the opensource code!! Could you tell me the meaning of metric in lr step? https://github.com/yitu-opensource/T2T-ViT/blob/f436fe4043069989ec5e0c2d07407b6d898493a7/main.py#L577-L579 https://github.com/yitu-opensource/T2T-ViT/blob/f436fe4043069989ec5e0c2d07407b6d898493a7/main.py#L688-L689
In my understanding.Look like in timm it's don't have special meaning.