search

MadryLab / constructed-datasets

Datasets for the paper "Adversarial Examples are not Bugs, They Are Features"
http://gradientscience.org/adv
185 stars 23 forks source link

Robust Accuracy on Cifar10 #8

Open SolidShen opened 4 years ago

SolidShen commented 4 years ago

Dear authors: In the paper (Table 7), you mentioned the robust accuracy on cifar10 for the standard training model is 4.49% (l2_bound = 0.25). Do you still use the same setting as you claimed in C.3 (7 steps with a step size of eps/5)? I am trying to reproduce your evaluation experiment but I can't achieve this accuracy on the same setting with the code from your lab released 'https://github.com/MadryLab/robustness'. (I can only get 11.3%) Could you provide more details about this experiment? Thanks!

andrewilyas commented 4 years ago

For evaluating the robustness, you can use more steps to lower the accuracy (also, 11.3% vs 4.49% could just be due to differences in model training/architecture/etc, so I would not worry too much about this)