Commands to reproduce the ViT-T and ViT-S experiments

OliverRensu / TinyMIM

151 stars 7 forks source link

Commands to reproduce the ViT-T and ViT-S experiments #5

Closed LiuDongyang6 closed 1 year ago

LiuDongyang6 commented 1 year ago

Thanks for this work!

Would you like to share the commands of the experiments with ViT-T and ViT-S as the student? I am interested in the MAE-VIT-B -> VIT-S and the MAE-VIT-B -> VIT-T experiments in Table 11, and I would like to know if any hyperparameters need to be changed.

OliverRensu commented 1 year ago

The pretraining commands are identical. During finetuning, we adpot a higher learning rate and larger batch size.