How the "kernel_ratio" influences the performance?

yitu-opensource / T2T-ViT

ICCV2021, Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet

Other

1.14k stars 177 forks source link

How the "kernel_ratio" influences the performance? #30

Closed PkuRainBow closed 3 years ago

PkuRainBow commented 3 years ago

Really nice work! I am wondering how the "kernel_ratio" of Token_performer influences the final performance.

https://github.com/yitu-opensource/T2T-ViT/blob/9dafdef17f6740fc600dad754e0f40bd246463e0/models/token_performer.py#L9

yuanli2333 commented 3 years ago

Hi, It has a few influences on the final performance when kernel_ration>=0.5. Based on our experiments, please don't use kernel_ration<0.5. We would update the analysis in our next arxiv verion.