search

ain-soph / trojanzoo

TrojanZoo provides a universal pytorch platform to conduct security researches (especially backdoor attacks/defenses) of image classification in deep learning.
https://ain-soph.github.io/trojanzoo
GNU General Public License v3.0
274 stars 62 forks source link

Hyperparameters for training Resnet18 on CIFAR10? #187

Closed kshitijsachan closed 1 year ago

kshitijsachan commented 1 year ago

Thanks for making this benchmark! You have provided great documentation.

In the trojanzoo paper, you have a table showing that you trained resnet18 on CIFAR10 with 95.37% accuracy? Do you have the hyper parameters for that training run? (e.g. did you use resnet18,resnet18_ap_comp, resnet18_comp, or resnet18_s, etc.) I've tried a few different configurations and have been unable to train a model with >90% accuracy.

I've also had trouble reproducing your results in Table 10 of the paper. I am generally getting better performance for STRIP and much worse performance on NEO than you reported, but I would guess this is because I am using different hyper parameters/models. Perhaps you still have the pretrained attacked model weights that you could share? Thanks.

ain-soph commented 1 year ago

python ./examples/train.py --color --verbose 1 --dataset cifar10 --model resnet18_comp --lr_scheduler --cutout --save