Closed ShengYun-Peng closed 12 months ago
Hi,
thanks for the submission, we'll add the model as soon as possible.
Thanks @fra31! Shall I commit to add_models_2 branch?
Either that or you can just add here the model definition (I guess the definition of all layers is already present in your code, so just the final architecture, similar to NormalizedWideResNet(...)
for CIFAR-10, would be needed), whatever it's more convenient for you.
The model is defined as slightly different from the CIFAR-10 WRN, but I'll add both models to the same file.
Hi @fra31, could you grant me access to the add_models_2 branch? I updated the code locally on my machine, but cannot push commits.
Can you maybe create a PR instead? I can take care of it from there.
Sure, PR created: https://github.com/RobustBench/robustbench/pull/153
Great, thanks!
Added the models with https://github.com/RobustBench/robustbench/pull/154. Is this preprocessing for ImageNet the right one?
Right, my test_transform is the same as Res256Crop224.
Just another question: are the reported clean and robust accuracy for ImageNet on the entire validation set (50k points) rather than on the subset (5k points) we use for the leaderboard?
It's just because for our 5k points subset I get 73.10% clean accuracy, while 73.448% for the full validation, which is much closer to (matches, depending on how one rounds) what you report. It might be also a difference in some package version though.
That makes sense.
It's just because for our 5k points subset I get 73.10% clean accuracy, while 73.448% for the full validation, which is much closer to (matches, depending on how one rounds) what you report. It might be also a difference in some package version though.
Do you mean that the clean accuracy is reported for the full validation set, while robust accuracy for the subset?
Oh, I meant the package differences. As shown here, I'm directly using the benchmark API from the robustbench, so there's no way the clean accuracy is on the full validation set, while the adversarial is only 5k.
The leaderboards should be updated.
Thanks so much! @fra31
Paper Information
Leaderboard Claim(s)
Add here the claim for your model(s). Copy and paste the following subsection for the number of models you want to add.
Model 1
Model 2
Model Zoo:
timm
.~ If not, I added the link to the architecture implementation so that it can be added.