search

xiangyu8 / SATA

6 stars 2 forks source link

Hyperparameters for TinyImageNet #1

Open tushaarkataria opened 2 weeks ago

tushaarkataria commented 2 weeks ago

Hello,

I am trying to re-create the results stated in your paper. I am following the hyper-parameters stated in the tiny image net folder, but that is consistently giving me models that are not converging. Can you confirm that you used the same hyper-parameters or did you use any different ones ?

python main.py --model vit --dataset T-IMNET --lr1 0.003

The hyperparameters for CIFAR100 are working fine, but for TinyImageNet it results in hugely overfitted models.

Regards Tushar

xiangyu8 commented 2 weeks ago

Hi Tushar, thanks for your interest in our work. I think you may have missed --lr2. You may try scripts under ./scripts/timagenet/vit/ fonder directly. Besides, hyper-parameters are shown in Table 1 in the paper.

tushaarkataria commented 2 weeks ago

Thanks for replying.

Isn't Lr2 only for parameters when using suppress and threshold? I cannot even recreate the base vit model's accuracy without using TWIST algorithm.

xiangyu8 commented 2 weeks ago

You may try https://github.com/aanna0701/SPT_LSA_ViT to reproduce ViT results.

tushaarkataria commented 2 weeks ago

I tried that also doesn't work. And it seems that other people had the same issue.

https://github.com/aanna0701/SPT_LSA_ViT/issues/14

Can you share any changes you made to run the TinyImageNet baseline?

tushaarkataria commented 1 week ago

Any information regarding this would be greatly appreciated. I am not able to recreate any results for the TinyImageNet dataset.