datasets preprare

[JIA-HONG-CHU]

I used the same as t2t_vit github (https://github.com/yitu-opensource/T2T-ViT) data prepare structure on white_box_test.py . and my command is python3 white_box_test.py --data_dir imagenet --mode foolbox --model vit_small_patch16_224 --pretrained True . But I got the awful acc .Below

sample size is : 1000 clean accuracy: 0.0 % Model vit_small_patch16_224 robust accuracy for LinfPGD perturbations with Step 40, Linf norm ≤ 0.001 : 0.0 % Step 40, Linf norm ≤ 0.003 : 0.0 % Step 40, Linf norm ≤ 0.005 : 0.0 % Step 40, Linf norm ≤ 0.008 : 0.0 % Step 40, Linf norm ≤ 0.01 : 0.0 % Step 40, Linf norm ≤ 0.1 : 0.0 %

Could u tell me why I got ? and what should I do? Tks u. :))

[RulinShao]

Hi @JIA-HONG-CHU,

Thanks for your interest in our code! The --model indicates which model you want to evaluate. So you may need to change the --model vit_small_patch16_224 to --model {T2T MODEL NAME} to evaluate the T2T-ViT model of your interest. As the timm package doesn't register T2T-ViT models by default, you need to follow the README.md in this GitHub link and include from models.t2t_vit import * in the script to load its models.

What you have run should return the evaluation results of vit_small_patch16_224, but the accuracies are all zeros now. We recommend you config your environment with our newly uploaded requirement file and run python3 white_box_test.py --data_dir imagenet --mode foolbox --model vit_small_patch16_224 again. If it works well for the vit_small_patch16_224, then it should also work well when you import the t2t_vit as described in here properly.

Hope this helps!

Best, Rulin

[RulinShao]

BTW, please make sure that you pass the path to the ImageNet dataset using --data_dir {PATH TO IMAGENET DATASET}. A double-check of the data loader is also preferred.

[JIA-HONG-CHU]

tks a lots . i will close this issue