About the result of ImageNet

[momo1986]

Hi Xiang,

I think you have done a wonder task.

I try the program.

However, I met some issues.

I follow your guidance to put the data and download the weights.

However, on the patchcure_vitsrf14x2_masked scenario, the result is not very obvious:

patch_size (96, 96) corruption_size (2, 4) feature_size (1, 7) mask_size (1, 4) mask_stride (1, 1) (1, 7) (1, 4) (1, 1) num of masks 1 4 Clean Accuracy: 0.00102 Certified Robust Accuracy: 0.00038

~

Does it indicate the potential bug?

Thanks & Regards! Momo

[xiangchong1]

I think this is the expected result. See Figure 7 of the paper https://arxiv.org/pdf/2310.13076 ViT14x2 is less robust to larger patches. In your experiment setting, the feature size is 7, the mask size is 4. The certification condition of the double-masking algorithm is to have two-mask correctness. i.e., the model prediction on all possible two-mask predictions should be correct. In this case, two masks could remove 4*2=8 features, which are larger than the entire feature map size. So it is expected to see zero robustness.