Question about run attack methods against multiple defenses

[CHR-ray]

I see your demo command 'Neural Cleanse against BadNet' case, but if I use this script, I will need to run the attack multiple times (how many defense methods then how many times I may need to run the attack process).

Is it possible to run the attack once and save some intermediate files to realize running the attack method only once? (Then use the generated attack intermediate files for multiple defense methods.)

Thank you!

[ain-soph]

Don't quite understand what you hope to achieve. Could you give an example? Is a bash script not suitable for the case? Or you can’t implement your own python script to test multiple defense?

[CHR-ray]

For example, if I want to run badnet against Neural cleanse, and strip.

What I understand is that if I use "python ./examples/backdoor_defense.py --color --verbose 1 --pretrained --validate_interval 1 --dataset cifar10 --model resnet18_comp --attack badnet --defense neural_cleanse --mark_random_init --epochs 50 --lr 0.01" and one other command for strip, then I may go through badnet attack twice and Neural cleanse once, and strip once. In this case, I think maybe we can run the badnet only once to save GPU running time. So I ask if it is possible only to run attack once and feed the result files to multiple defense methods to save GPU running time.

Is my understanding of your code right? Is this possible? (My question is only about efficiency, of course, you demo is sufficient to run multiple attacks against multiple defense methods.)

[ain-soph]

backdoor_defense.py requires the previously saved attack results. It won’t run attack and it’s barely defense.

You need to run backdoor_attack.py before only once (for one attack setting).

[CHR-ray]

Thank you very much. I may misunderstand your code. But I do not see a term in your command to input the location of attack results, so how do you know where is the attack result?

[ain-soph]

@CHR-ray https://github.com/ain-soph/trojanzoo/blob/main/trojanzoo/configs/attack.yml

Both backdoor_attack.py and backdoor_defense.py will use the config settings attack_dir to save and load attack results.

So it is for data_dir and model_dir

You could refer https://ain-soph.github.io/trojanzoo/trojanvision/configs.html#module-trojanvision.configs for more details about how to set up different configs.

[ain-soph]

configs.zip For example, this is my personal configs. I put it under the same directory where I execute commands, so it serves as the project configs.

You could also set a global user configs following the docs.

[CHR-ray]

I see, this question related to #153 . I understand. Thank you.