Can you share which datasets are used to pretrain BiT models?

MetaMain / ViTRobust

Code corresponding to the paper: "On the Robustness of Vision Transformers": https://arxiv.org/abs/2104.02610

BSD 3-Clause "New" or "Revised" License

23 stars 7 forks source link

Can you share which datasets are used to pretrain BiT models? #4

Open gihyunkim opened 2 years ago

gihyunkim commented 2 years ago

I tried to attack BiT and ViT (both from timm pretrained imagenet-21k) by FGSM and I got different results from paper. I have no idea what differences are between this model and that situation.

I tried epsilon 0.1/255, 0.5/255, 1/255, 4/255, 8/255 and ViT does vulnerable than BiTM-101 but not BiTM-50. Datasets: Neurips dataset

timm: https://github.com/rwightman/pytorch-image-models

MetaMain commented 2 years ago

Hi! Thank you for your interest in our work. Can you pleas clarify what dataset you are using, the specific model BiT or ViT model you are testing, and which result you are trying to recreate from our paper?