Could you tell us whether we have reproduced the proposed method.

[JohnCarpenter2]

Thanks for open-sourcing your wonderful work.

We have trained the MMD kernel and used the following code to evaluate the performance:

CUDA_VISIBLE_DEVICES=0
python eval_epsad.py --detection_datapath './score_diffusion_t_cifar_stand' \
    --num_sub 500 \
    --adv_batch_size 500 \
    --epsilon 0.01569 \
    --diffuse_t 20 \
    --single_vector_norm_flag \
    --perb_image

The exp result shows that:

attack_method: FGSM, robust accuracy: top1:0.35200000762939454--top5:0.8220000457763672
attack and diffuison time: 24.67634677886963
score_tensor.shape:torch.Size([500, 3, 32, 32])
attack_method: PGD, robust accuracy: top1:0.058000001907348636--top5:0.5020000076293946
attack and diffuison time: 19.34290313720703
score_tensor.shape:torch.Size([500, 3, 32, 32])
attack_method: BIM, robust accuracy: top1:0.045999999046325686--top5:0.40799999237060547
attack and diffuison time: 19.499910831451416
score_tensor.shape:torch.Size([500, 3, 32, 32])
attack_method: MIM, robust accuracy: top1:0.08199999809265136--top5:0.45799999237060546
attack and diffuison time: 19.615848064422607
score_tensor.shape:torch.Size([500, 3, 32, 32])
attack_method: TIM, robust accuracy: top1:0.25--top5:0.76
attack and diffuison time: 19.774481534957886
score_tensor.shape:torch.Size([500, 3, 32, 32])
attack_method: CW, robust accuracy: top1:0.04400000095367432--top5:0.8440000152587891
attack and diffuison time: 22.0277259349823
score_tensor.shape:torch.Size([500, 3, 32, 32])
attack_method: DI_MIM, robust accuracy: top1:0.8780000305175781--top5:0.9820000457763672
attack and diffuison time: 19.90054965019226
score_tensor.shape:torch.Size([500, 3, 32, 32])
attack_method: FGSM_L2, robust accuracy: top1:0.3820000076293945--top5:0.85
attack and diffuison time: 18.244937896728516
score_tensor.shape:torch.Size([500, 3, 32, 32])
attack_method: PGD_L2, robust accuracy: top1:0.026000001430511475--top5:0.26399999618530273
attack and diffuison time: 19.957834243774414
score_tensor.shape:torch.Size([500, 3, 32, 32])
attack_method: BIM_L2, robust accuracy: top1:0.03799999952316284--top5:0.3220000076293945
attack and diffuison time: 19.931138038635254
score_tensor.shape:torch.Size([500, 3, 32, 32])

We are doubt whether we have reproduced the proposed method.

Hope for your response.

Thanks a lot.

[ZSHsh98]

Thanks a lot for drawing attention to our work!

The printed results seem correct for all 12 attack methods. In our code, we first generate the EPSs of adversarial samples and natural samples, then detect them using the trained deep kernel MMD subsequently. The results mentioned above are the processes of generating EPSs of adversarial samples, where "robust acc: top1--top5" is the accuracy of the standard classifier against adversarial samples. If you would like to obtain the final detection accuracy of our EPS-AD, please continue to run the remaining procedures in the second step of our repositories.

Hope the answer above can help you.