Accuracy for Common Corruptions.

[wjun0830]

Hi Hendrycks! May I ask one more thing? Your reply to Adversarial helped me a lot.

1. Can you tell me how to change corruption strengths for CIFAR-10-C? I downloaded Cifar-10-C from the link provided in https://github.com/hendrycks/robustness. However, there seems to be nothing related to corruption strengths.

2. Can you provide your exact performance in common corruption setting? And Did you report best accuracy? or the accuracy from the last epoch?

Since if Cifar-10-C dataset contain all five corruption strengths, then the epoch 86 seems to be most similar with your results. However, clean accuracy doesn't seem to reach 95.5% at all with WRN40-2. Can you provide how to reproduce your settings?

corrupt acc :  [53.946000000000005, 64.344, 57.306000000000004, 67.526, 58.382, 84.642, 79.41, 79.276, 71.054, 79.024, 83.966, 83.89, 93.55799999999999, 91.96600000000001, 82.48, 73.512, 83.636, 87.90599999999999, 81.658]
Epoch  86 |Time   150 |Tr Loss 0.0477 |Te Loss 0.884 |Test acc  |corrupt mean acc 76.71

corrupt acc :  [51.488, 62.674, 57.668, 66.06, 57.348, 84.75399999999999, 78.518, 78.544, 71.41199999999999, 78.74799999999999, 84.206, 85.49600000000001, 93.95599999999999, 92.78999999999999, 84.552, 72.418, 83.418, 87.768, 80.138]
Epoch 100 |Time   150 |Tr Loss 0.0261 |Te Loss 1.010 |Test acc  |corrupt mean acc 76.42

Thanks a lot, again!

[wjun0830]

Sorry clean accuracy problem was from my mistake. Can you give me a little help for just above questions?

[hendrycks]

However, there seems to be nothing related to corruption strengths.

The first 10K have the lowest corruption strength, and the last 10K examples have the highest corruption strength.

or the accuracy from the last epoch?

Since I use the cosine learning rate, I just use the model at the last epoch.