RobustBench / robustbench

RobustBench: a standardized adversarial robustness benchmark [NeurIPS 2021 Benchmarks and Datasets Track]
https://robustbench.github.io
Other
669 stars 99 forks source link

Pickle error for the cifar10 Standard model #152

Closed liuyuezhang closed 1 year ago

liuyuezhang commented 1 year ago

I encountered errors while trying to run this code

from robustbench.utils import load_model

model = load_model(model_name='Standard', dataset='cifar10', threat_model='Linf')

The error information is listed as follows:

Files already downloaded and verified
---------------------------------------------------------------------------
UnpicklingError                           Traceback (most recent call last)
Cell In[1], line 7
      3 x_test, y_test = load_cifar10(n_examples=50)
      5 from robustbench.utils import load_model
----> 7 model = load_model(model_name='Standard', dataset='cifar10', threat_model='Linf')

File [~/anaconda3/envs/pytorch/lib/python3.9/site-packages/robustbench/utils.py:123](https://vscode-remote+ssh-002dremote-002bweixx1.vscode-resource.vscode-cdn.net/home/lyz/Git/recurrent-denoise/~/anaconda3/envs/pytorch/lib/python3.9/site-packages/robustbench/utils.py:123), in load_model(model_name, model_dir, dataset, threat_model, norm)
    121 if not os.path.isfile(model_path):
    122     download_gdrive(models[model_name]['gdrive_id'], model_path)
--> 123 checkpoint = torch.load(model_path, map_location=torch.device('cpu'))
    125 if 'Kireev2021Effectiveness' in model_name or model_name == 'Andriushchenko2020Understanding':
    126     checkpoint = checkpoint['last']  # we take the last model (choices: 'last', 'best')

File [~/anaconda3/envs/pytorch/lib/python3.9/site-packages/torch/serialization.py:815](https://vscode-remote+ssh-002dremote-002bweixx1.vscode-resource.vscode-cdn.net/home/lyz/Git/recurrent-denoise/~/anaconda3/envs/pytorch/lib/python3.9/site-packages/torch/serialization.py:815), in load(f, map_location, pickle_module, weights_only, **pickle_load_args)
    813     except RuntimeError as e:
    814         raise pickle.UnpicklingError(UNSAFE_MESSAGE + str(e)) from None
--> 815 return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)

File [~/anaconda3/envs/pytorch/lib/python3.9/site-packages/torch/serialization.py:1033](https://vscode-remote+ssh-002dremote-002bweixx1.vscode-resource.vscode-cdn.net/home/lyz/Git/recurrent-denoise/~/anaconda3/envs/pytorch/lib/python3.9/site-packages/torch/serialization.py:1033), in _legacy_load(f, map_location, pickle_module, **pickle_load_args)
   1027 if not hasattr(f, 'readinto') and (3, 8, 0) <= sys.version_info < (3, 8, 2):
   1028     raise RuntimeError(
   1029         "torch.load does not work with file-like objects that do not implement readinto on Python 3.8.0 and 3.8.1. "
   1030         f"Received object of type \"{type(f)}\". Please update to Python 3.8.2 or newer to restore this "
   1031         "functionality.")
-> 1033 magic_number = pickle_module.load(f, **pickle_load_args)
   1034 if magic_number != MAGIC_NUMBER:
   1035     raise RuntimeError("Invalid magic number; corrupt file?")

UnpicklingError: invalid load key, '<'.

I can however download other models on CIFAR10 normally. Any idea for this problem? Thanks.

I am using python 3.9, Pytorch 2.0.0, cuda 11.0.

fra31 commented 1 year ago

Hi,

can you please try to delete the checkpoint and load the model again (it might be corrupted)?

liuyuezhang commented 1 year ago

Sure. Shall I just delete the Standard.pt checkpoint under 'models/cifar10/Linf'? If that's the case then I have already redo it. Still the same problem. I can however load other models such as Carmon2019Unlabeled.pt normally.

A bit weird.

fra31 commented 1 year ago

Yes, deleting the checkpoint should be enough. I just tested it and could download the model normally.

liuyuezhang commented 1 year ago

Dear fra31,

I retried it. The problem still persists. Is it possible for us to have a short zoom session for me to show you the problem?

Thanks!

Best, Yuezhang

On Sun, Sep 3, 2023 at 12:02 PM fra31 @.***> wrote:

Yes, deleting the checkpoint should be enough. I just tested it and could download the model normally.

— Reply to this email directly, view it on GitHub https://github.com/RobustBench/robustbench/issues/152#issuecomment-1704353596, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADTXFRFMGTJFE54H6MIQHNDXYSZZ3ANCNFSM6AAAAAA4I77JOU . You are receiving this because you authored the thread.Message ID: @.***>

--

Liu, Yuezhang

PhD student in Neuroscience

University of Texas at Austin

liuyuezhang commented 1 year ago

I think I found the problem. The Standard.pt must be corrupted, as it only has 2.3KB size. The contents show in the file

<!DOCTYPE html><html><head><title>Google Drive - Virus scan warning</title><meta http-equiv="content-type" content="text/html; charset=utf-8"/><style nonce="qGEOwx0JRS22ICTsisresA">.goog-inline-block{position:relative;display:-moz-inline-box;display:inline-block}* html .goog-inline-block{display:inline}*:first-child+html .goog-inline-block{display:inline}.goog-link-button{position:relative;color:#15c;text-decoration:underline;cursor:pointer}.goog-link-button-disabled{color:#ccc;text-decoration:none;cursor:default}body{color:#222;font:normal 13px/1.4 arial,sans-serif;margin:0}.grecaptcha-badge{visibility:hidden}.uc-main{padding-top:50px;text-align:center}#uc-dl-icon{display:inline-block;margin-top:16px;padding-right:1em;vertical-align:top}#uc-text{display:inline-block;max-width:68ex;text-align:left}.uc-error-caption,.uc-warning-caption{color:#222;font-size:16px}#uc-download-link{text-decoration:none}.uc-name-size a{color:#15c;text-decoration:none}.uc-name-size a:visited{color:#61c;text-decoration:none}.uc-name-size a:active{color:#d14836;text-decoration:none}.uc-footer{color:#777;font-size:11px;padding-bottom:5ex;padding-top:5ex;text-align:center}.uc-footer a{color:#15c}.uc-footer a:visited{color:#61c}.uc-footer a:active{color:#d14836}.uc-footer-divider{color:#ccc;width:100%}sentinel{}</style><link rel="icon" href="//ssl.gstatic.com/docs/doclist/images/drive_2022q3_32dp.png"/></head><body><div class="uc-main"><div id="uc-dl-icon" class="image-container"><div class="drive-sprite-aux-download-file"></div></div><div id="uc-text"><p class="uc-warning-caption">Google Drive can't scan this file for viruses.</p><p class="uc-warning-subcaption"><span class="uc-name-size"><a href="/open?id=1t98aEuzeTL8P7Kpd5DIrCoCL21BNZUhC">natural.pt.tar</a> (278M)</span> is too large for Google to scan for viruses. Would you still like to download this file?</p><form id="download-form" action="https://docs.google.com/uc?export=download&amp;id=1t98aEuzeTL8P7Kpd5DIrCoCL21BNZUhC&amp;confirm=t&amp;uuid=341f39f4-c391-4779-94f4-8f4933f7f292" method="post"><input type="submit" id="uc-download-link" class="goog-inline-block jfk-button jfk-button-action" value="Download anyway"/></form></div></div><div class="uc-footer"><hr class="uc-footer-divider"></div></body></html>

Seems to be some virus canning warning.

Best, Yuezhang

fra31 commented 1 year ago

Ok. Please find here the notebook where I tested it, in case it helps.

liuyuezhang commented 1 year ago

Thanks. Do you mind providing a public sharing link for the Standard.pt (Cifar 10, Linf) directly? In this case, I can download it directly.

It seems to be a common problem for downloading with Google drive on large files in some system configuration, such as https://github.com/tensorflow/datasets/issues/3935

Thanks!

fra31 commented 1 year ago

See here for a pointer to all the models.