search

fra31 / auto-attack

Code relative to "Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks"
https://arxiv.org/abs/2003.01690
MIT License
639 stars 111 forks source link

Regarding the checks recently added to AutoAttack #73

Closed zangobot closed 2 years ago

zangobot commented 2 years ago

Good evening!

I was browsing the repository, and I found this page with potential checks that can be applied to better evaluate attacks (https://github.com/fra31/auto-attack/blob/master/flags_doc.md) We worked on a similar topic, and we published a preprint in June (https://arxiv.org/abs/2106.09947) where we develop indicators that trigger when an evaluation is faulty (and among these indicators, there is also the zero-gradient check). In the process, we also evaluated AutoPGD, showing that our systematic checks can patch failures that the automatic algorithm is unable to find. We also released the code of our research on GitHub (https://github.com/pralab/IndicatorsOfAttackFailure).

It would be great if you could add a reference to our paper/code to that page, as the underlying idea is essentially very similar. Hence, interested users can benefit from both sources as well.

Thank you in advance!

fra31 commented 2 years ago

Hi,

thanks for mentioning your relevant work, I'll add a reference to it!

zangobot commented 2 years ago

Thank you! Looking forward to it!

fra31 commented 2 years ago

Added with https://github.com/fra31/auto-attack/commit/fd09b96d10d39df5767e056dea8ab986f884afb2.

zangobot commented 2 years ago

Really thank you!!!