search

fra31 / auto-attack

Code relative to "Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks"
https://arxiv.org/abs/2003.01690
MIT License
639 stars 111 forks source link

ensemble attack #85

Open Jialiang14 opened 2 years ago

Jialiang14 commented 2 years ago

Hello, your job is great! I am confused about one point. If one image could not be perturbed successfully by attacker A, it would be perturbed by attacker B. Then I have a question, attacker B would perturb the original image or the perturbed image by attacker A?

fra31 commented 2 years ago

Hi,

thanks! Each attack uses the original (unperturbed) image as target point.

Jialiang14 commented 2 years ago

image

Hello, I use the autoattack with standard and non-individual version. It outputs four robust accuracies under four attacks. Which robust accuracy should I choose at last as the evaluation of the model?

fra31 commented 2 years ago

Hi,

if you're using run_standard_evaluation you should take the last one, which includes all attacks. If instead you use run_standard_evaluation_individual you need to compute the worst-case over the different methods manually.

Hope this helps!