Closed tongwu2020 closed 11 months ago
Hi @tongwu2020. Can you try removing --drop-tokens
?
Hi Hadi,
Thanks for the quick reply. I tried removing the --drop-tokens
and using the same pretrained model you provided.
Here are my results.
Clean Acc | Certified Robustness | |
---|---|---|
Drop-tokens | 68.83% | 34.52% |
No Drop-tokens | 69.69% | 39.13% |
Reported | 69.3% | 38.3% |
I wonder whether the reported results in Table 1 are dorp-tokens or not dorp-tokens?
Thank you so much. Tong
Yeah the results in Table 1 are all without --drop-token. So now you are getting closer results to what we report. I wonder if the slight increase in what you are getting might be an issue of a different PyTorch version? Unfortunately, I can't find what PyTorch version we used for this. But other than that, all the hyperparameters you are using look good to me.
Thank you so much for your help. Have a good day.
Hi, thanks for your code and great paper. I am trying to replicate the results from the paper.
I am using this command to replicate the results from Table 1 (ViT-B (b = 19) and 2% pixels).
I get 68.83% clean acc and 34.52% certified robustness, whereas the report one is 69.3% and 38.3%. Did I make some mistakes?
Thank you so much.