RobustBench / robustbench

RobustBench: a standardized adversarial robustness benchmark [NeurIPS 2021 Benchmarks and Datasets Track]
https://robustbench.github.io
Other
671 stars 99 forks source link

About Test Augmentation #115

Closed XixuHu closed 1 year ago

XixuHu commented 1 year ago

Hi, I am trying to use Robustbench to test the accuracy of my model. But I came across a data augmentation problem. I trained my model using the dataset mean and std in the transform, but I notice that there is not a corresponding transform in the test data loading in Robustbench. Thus the model I test with dataset mean and std transform can enjoy like 80% acc, but it drops to 20% something in the Robustbench clean_accuracy test. What should I do to make my models enjoy the same clean test acc in RobustBench? Because if I add the dataset mean and std transform in it, the bounds=(0,1) in the following attack test will be invalid. I am new to this field, so any suggestions are warmly welcome! Thank you so much!

XixuHu commented 1 year ago

I use the from robustbench.model_zoo.architectures.utils_architectures import normalize_model to solve my problem.

dedeswim commented 1 year ago

Yes, that's the way to go! I'm glad it worked out