I read your paper “For models trained from scratch an initial learning rate of 5e−4 with a cosine learning rate schedule [26] is adopted” But how should I implement it in the code?

[noahzn]

HI, I have shared the source code in this repo. You don't need to implement by yourself.

[3ming]

But I did not add pre-trained weights, set epochs to 35, learning rate to --lr 0.0005 5e-6 31 0.0001 1e-5 31, and in the end, I could not achieve the effect of the paper with δ < 1.25, only reaching 0.86

[noahzn]

In our paper without using pre-trained weights, δ < 1.25 is 0.859. Your 0.86 is already better than the number reported in the paper.

[3ming]

Okay, thank you!