search

raoyongming / DynamicViT

[NeurIPS 2021] [T-PAMI] DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification
https://dynamicvit.ivg-research.xyz/
MIT License
551 stars 69 forks source link

Token keep ratio setting #45

Open King4819 opened 4 months ago

King4819 commented 4 months ago

Hi, I want to ask that is there any specific reason for the token keep ratio setting ?

[1, 1, 1, 0.7, 0.7, 0.7, 0.7^2, 0.7^2, 0.7^2, 0.7^3, 0.7^3, 0.7^3] for DeiT 12 layers

Is it related to prune FLOPs ratio, such as keep ratio 0.7 will get approximately prune FLOPs ratio of 30%

Thanks !!!