Closed shuuchen closed 1 year ago
Hi,
Have you implemented cosine learning rate annealing as talked in the paper?
Thank you!
Yes, you could see https://github.com/keyu-tian/SparK/blob/main/pretrain/utils/lr_control.py#L11. And this function is only called in https://github.com/keyu-tian/SparK/blob/main/pretrain/main.py#L146.
Hi,
Have you implemented cosine learning rate annealing as talked in the paper?
Thank you!