search

SHI-Labs / Compact-Transformers

Escaping the Big Data Paradigm with Compact Transformers, 2021 (Train your Vision Transformers in 30 mins on CIFAR-10 with a single GPU!)
https://arxiv.org/abs/2104.05704
Apache License 2.0
495 stars 77 forks source link

interpolation of imagenet #41

Closed JingyangXiang closed 2 years ago

JingyangXiang commented 2 years ago

Hi, sorry to trouble you again. I have successfully your results on cifar10, and I plan to reproduce the results on ImageNet. I follow your configs for imagenet and my result is lower than yours about 2%.

I would like to know the interpolation's type used in your paper for imagenet. As most paper's only use Bicubic, and your paper use "random" for imagenet.

Thanks very much!

alihassanijr commented 2 years ago

Hi, I'm not sure if that's the issue here. Can you share your logs? Also, can you verify your ImageNet copy's MD5?

JingyangXiang commented 2 years ago

https://drive.google.com/file/d/1XuQTQtFKY2qZfMWJxzoPBOKZNYoqezAF/view?usp=sharing Hi, I upload my log to google drive, I hope I can compare it with your log. Thanks very much!

alihassanijr commented 2 years ago

I'm sorry, this log looks very different from the logs our training script provides. Are you using the same training script? Can you share the full command as well? Also, this logs seems to include only a limited number of epochs. The learning rate also doesn't match the config file.

JingyangXiang commented 2 years ago

Thanks, I have found the difference, for I make lr = batch_size device_num / 512 lr, I will fix it and try again. Thanks for your help!