search

raoyongming / DynamicViT

[NeurIPS 2021] [T-PAMI] DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification
https://dynamicvit.ivg-research.xyz/
MIT License
576 stars 72 forks source link

DeiT lr question #42

Closed King4819 closed 11 months ago

King4819 commented 11 months ago

Excuse me, I want to ask that: why your training cmds of deit is "--lr 1e-3", but the default settings in your code is "4e-3", Thanks !

raoyongming commented 11 months ago

Hi @King4819, our new codebase combined the training code of DeiT and ConvNeXt. The "4e-3" learning rate is for ConvNeXt and Swin models.