LiuHC0428 / LAW-GPT

中文法律对话语言模型
1.06k stars 119 forks source link

项目启动失败,OSError: Unable to load weights from pytorch checkpoint file for 'model/pytorch_model-00001-of-00008.bin' #27

Closed hycao closed 1 year ago

hycao commented 1 year ago

已经按照 README 中下载配置了

已经检查了 pip 安装软件的版本都和 requirements.txt 中一致

机器配置 PyTorch 1.11.0 Python 3.8(ubuntu20.04) Cuda 11.3 GPU RTX 3090(24GB) * 1

使用 CUDA_VISIBLE_DEVICES=$cuda_id python ./demo.py 启动

提示如下错误:

root@autodl-container-1ee1119b3c-b7155cb8:~/LAW-GPT/src# CUDA_VISIBLE_DEVICES=$cuda_id python ./demo.py
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Loading checkpoint shards:   0%|                                                                                                             | 0/8 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/modeling_utils.py", line 415, in load_state_dict
    return torch.load(checkpoint_file, map_location="cpu")
  File "/root/miniconda3/lib/python3.8/site-packages/torch/serialization.py", line 777, in load
    with _open_zipfile_reader(opened_file) as opened_zipfile:
  File "/root/miniconda3/lib/python3.8/site-packages/torch/serialization.py", line 282, in __init__
    super(_open_zipfile_reader, self).__init__(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/modeling_utils.py", line 419, in load_state_dict
    if f.read(7) == "version":
  File "/root/miniconda3/lib/python3.8/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 64: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./demo.py", line 148, in <module>
    main()
  File "./demo.py", line 77, in main
    model = model_class.from_pretrained(args.model_path, device_map = device_map).half()
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/modeling_utils.py", line 2478, in from_pretrained
    ) = cls._load_pretrained_model(
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/modeling_utils.py", line 2780, in _load_pretrained_model
    state_dict = load_state_dict(shard_file)
  File "/root/miniconda3/lib/python3.8/site-packages/transformers/modeling_utils.py", line 431, in load_state_dict
    raise OSError(
OSError: Unable to load weights from pytorch checkpoint file for 'model/pytorch_model-00001-of-00008.bin' at 'model/pytorch_model-00001-of-00008.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.

请问有人知道是什么问题导致的吗?感谢感谢

hycao commented 1 year ago

看了下,好像 model 文件的大小和实际大小不一致,我删除了重新上传一次