throw VE when attacks to run is specified but version is not 'custom'

fra31 / auto-attack

Code relative to "Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks"

https://arxiv.org/abs/2003.01690

MIT License

661 stars 113 forks source link

throw VE when attacks to run is specified but version is not 'custom' #80

Closed niklasnolte closed 3 years ago

niklasnolte commented 3 years ago

it was quite confusing to me that when i specified attacks to run, it would just overwrite that unless i also specified another argument.

fra31 commented 3 years ago

Hi,

thanks for the contribution! Sounds a good idea to me. However, the attacks get overridden if the version is one of those predefined here https://github.com/fra31/auto-attack/blob/6482e4d6fbeeb51ae9585c41b16d50d14576aadc/autoattack/autoattack.py#L65 Then, I'd replace version != 'custom' with version in <list of versions> as condition to raise the VE. What do you think?

niklasnolte commented 3 years ago

agreed :) is there any other version besides custom that does not override attacks_to_run right now? in this case i would edit the error message

fra31 commented 3 years ago

Any version which is non predefined shouldn't override it. The idea was to leave the option to each user of adding new versions, on top of the default ones.