search

Azure / counterfit

a CLI that provides a generic automation layer for assessing the security of ML models
MIT License
805 stars 129 forks source link

Parsing issue in scan. #9

Closed erickgalinkin closed 3 years ago

erickgalinkin commented 3 years ago

When attempting to scan using the -a option with multiple attacks, the script only launches the last attack.

I am using a tensorflow model trained on mnist. I've attached images displaying the issue. The issue occurs with both the "image" and "numpy" model_data_type.

Screen Shot 2021-05-20 at 9 59 57 AM

Screen Shot 2021-05-20 at 9 56 18 AM

moohax commented 3 years ago

Thanks Eric.

Found that we added nargs+ to --attack but didn't implement the logic to actually execute on every attack...

moohax commented 3 years ago

This has been fixed in the competition branch (docker run -p8888:8888 azuretrustworthyml/counterfit-mlsecevasion:latest). We will backport to main, but this has some additional attacks that you would probably be interested in.