search

hpcaitech / ColossalAI

Making large AI models cheaper, faster and more accessible
https://www.colossalai.org
Apache License 2.0
38.59k stars 4.32k forks source link

[BUG]: pickle_module.load has bug: _pickle.UnpicklingError: invalid load key, 'v'. #3511

Open Modas-Li opened 1 year ago

Modas-Li commented 1 year ago

🐛 Describe the bug

Loading checkpoint shards: 0%| | 0/33 [00:00<?, ?it/s] Traceback (most recent call last): File "/data/anaconda3/lib/python3.7/site-packages/transformers/modeling_utils.py", line 415, in l oad_state_dict return torch.load(checkpoint_file, map_location="cpu") File "/data/anaconda3/lib/python3.7/site-packages/torch/serialization.py", line 795, in load return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args) File "/data/anaconda3/lib/python3.7/site-packages/torch/serialization.py", line 1002, in _legacy_ load magic_number = pickle_module.load(f, **pickle_load_args) _pickle.UnpicklingError: invalid load key, 'v'.

Environment

No response

JThh commented 1 year ago

Hi, what was your command to run the codes, and where did you download the checkpoints?