[Bug] Getting pickle error upon loading existing model

[giladcohen]

Running:

from robustbench.data import load_cifar10
from robustbench.utils import load_model
x_test, y_test = load_cifar10(n_examples=50, data_dir='/data/dataset/cifar10')
model = load_model(model_name='Carmon2019Unlabeled', dataset='cifar10', threat_model='Linf')

returns:

  File "/home/gilad/venv_py37_new/lib/python3.7/site-packages/torch/serialization.py", line 777, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '<'.

my pip list output is attached. pip_list.txt

[max-andr]

There seems to be some issue with the automatic downloading from Google Drive (see e.g. here). We are looking into this!

[fra31]

It should still be possible to download the models via browser with https://drive.google.com/uc?id=<gdrive_id>, where <gdrive_id> can be read in model_zoo/cifar10.py (e.g. here).

[max-andr]

should be resolved now with the latest commit! sorry again. feel free to reopen the issue if the problem persists.

[LucasFenaux]

Hi.

It appears the error is back.

Running:

from robustbench.data import load_cifar10 x_test, y_test = load_cifar10(n_examples=50) from robustbench.utils import load_model model = load_model(model_name='Carmon2019Unlabeled', threat_model='Linf')

returns:

UnpicklingError Traceback (most recent call last) in <cell line: 5>() 3 4 from robustbench.utils import load_model ----> 5 model = load_model(model_name='Carmon2019Unlabeled', threat_model='Linf') 2 frames /usr/local/lib/python3.10/dist-packages/torch/serialization.py in _legacy_load(f, map_location, pickle_module, pickle_load_args) 1244 "functionality.") 1245 -> 1246 magic_number = pickle_module.load(f, pickle_load_args) 1247 if magic_number != MAGIC_NUMBER: 1248 raise RuntimeError("Invalid magic number; corrupt file?")

UnpicklingError: invalid load key, '<'.

This error was produced using your quickstart notebook on google colab. I also got the same error when trying to load models on my personal machine.

[fra31]

Hi,

I'm not sure what the problem is, but it seems related to https://github.com/wkentaro/gdown/issues/291. Here https://github.com/RobustBench/robustbench/tree/fix-download is a fix following https://github.com/wkentaro/gdown/pull/294, please let me know if it works for you.

[LucasFenaux]

That seems to have done the trick. Thank you.

[fra31]

It should be fixed with https://github.com/RobustBench/robustbench/pull/164, feel free to reopen if not.

[amirhagai]

I'm experiencing the same issue.

Executing the following code:

from robustbench.model_zoo.cifar10 import linf from robustbench.utils import load_model

errors = []

for key in linf: try: model = load_model(key, dataset='cifar10', threat_model='Linf') except Exception as e:

This helps identify the type of error occurring

    print("The error is:", e)
    errors.append(key)
    print(f"Key error - {key}")

print("Number of models loaded successfully:", len(linf) - len(errors))

Only 8 models are successfully loaded, with all errors being identical: "The error is: invalid load key, '<'."

[fra31]

Hi,

https://github.com/RobustBench/robustbench/pull/175 should fix this, could you please let me know if it works for you (you'll have to install the branch version)?

[amirhagai]

Thanks a lot ! :)

All seems to work except of

Sehwag2021Proxy The error is: No module named 'easydict'

[fra31]

It's a bit weird, I just tried and it seems to work fine. Also, it doesn't seem an issue related to downloading.

[amirhagai]

Hi! Sorry for the late response, I was away. Maybe it's happens as i"m working with Windows. I try to install the branch at Colab and it works.

Anyway, thanks a lot again :)