Closed ssssholmes closed 2 years ago
The largest learning rate for stable training in this codebase is 0.001.
Thanks for your great work. But the paper says:"We use the stochastic depth (Huang et al., 2016) with a rate of 10% to reduce overfitting." Have you tried to use stochastic depth method in your model?
I will try stochastic depth in the future.
The authors mentioned they use adamw with 0.003 learning rate without imagenet pretrain. I tried it and find it is really hard to train with such config.