search

SkyworkAI / Skywork

Skywork series models are pre-trained on 3.2TB of high-quality multilingual (mainly Chinese and English) and code data. We have open-sourced the model, training data, evaluation data, evaluation methods, etc. 天工系列模型在3.2TB高质量多语言和代码数据上进行预训练。我们开源了模型参数，训练数据，评估数据，评估方法。
Other
1.21k stars 111 forks source link

OSError: Unable to load weights from pytorch checkpoint file for Skywork-13B-base/pytorch_model-00010-of-00053.bin' If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True. #21

Closed bolongliu closed 10 months ago

bolongliu commented 10 months ago

已解决，主要原因是下载模型过程中没有下载完全，导致下载文件破损而出现加载报错。查看是否完整可以使用如下命令。

ll

(base) ➜  Skywork-13B-base git:(main) ✗ ll
总用量 26G
-rw-rw-r-- 1 liubl liubl  733 Nov  1 21:58  config.json
-rw-rw-r-- 1 liubl liubl 3.1K Nov  1 21:58  configuration_skywork.py
-rw-rw-r-- 1 liubl liubl  182 Nov  1 21:58  generation_config.json
drwxrwxr-x 2 liubl liubl 4.0K Nov  1 21:58  misc
-rw-rw-r-- 1 liubl liubl  39K Nov  1 21:58  modeling_skywork.py
-rw-rw-r-- 1 liubl liubl 487M Nov  2 13:07  pytorch_model-00001-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  1 23:43  pytorch_model-00002-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 13:35  pytorch_model-00003-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 13:40  pytorch_model-00004-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 13:40  pytorch_model-00005-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 13:40  pytorch_model-00006-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 13:38  pytorch_model-00007-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 13:35  pytorch_model-00008-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 13:38  pytorch_model-00009-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 17:15  pytorch_model-00010-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 17:21  pytorch_model-00011-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  1 23:44  pytorch_model-00012-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  1 23:41  pytorch_model-00013-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  1 23:42  pytorch_model-00014-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  1 23:42  pytorch_model-00015-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  1 23:42  pytorch_model-00016-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  1 23:43  pytorch_model-00017-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  1 23:41  pytorch_model-00018-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  1 23:42  pytorch_model-00019-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  1 23:43  pytorch_model-00020-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 13:54  pytorch_model-00021-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 13:57  pytorch_model-00022-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 13:58  pytorch_model-00023-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 13:59  pytorch_model-00024-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 13:59  pytorch_model-00025-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 13:59  pytorch_model-00026-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 14:00  pytorch_model-00027-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 14:00  pytorch_model-00028-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 14:00  pytorch_model-00029-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 14:00  pytorch_model-00030-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 14:19  pytorch_model-00031-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 14:21  pytorch_model-00032-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 14:21  pytorch_model-00033-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 14:22  pytorch_model-00034-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 14:22  pytorch_model-00035-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 14:22  pytorch_model-00036-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 14:22  pytorch_model-00037-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 14:23  pytorch_model-00038-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 14:23  pytorch_model-00039-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 14:23  pytorch_model-00040-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 14:24  pytorch_model-00041-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 15:08  pytorch_model-00042-of-00053.bin
-rw-rw-r-- 1 liubl liubl    0 Nov  2 15:25  pytorch_model-00043-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 15:06  pytorch_model-00044-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 15:06  pytorch_model-00045-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 15:07  pytorch_model-00046-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 15:07  pytorch_model-00047-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 15:07  pytorch_model-00048-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 15:07  pytorch_model-00049-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 15:07  pytorch_model-00050-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 15:07  pytorch_model-00051-of-00053.bin
-rw-rw-r-- 1 liubl liubl 487M Nov  2 15:07  pytorch_model-00052-of-00053.bin
-rw-rw-r-- 1 liubl liubl 1.2G Nov  2 15:10  pytorch_model-00053-of-00053.bin
-rw-rw-r-- 1 liubl liubl  41K Nov  1 21:58  pytorch_model.bin.index.json
-rw-rw-r-- 1 liubl liubl  22K Nov  1 21:58  README.md
-rw-rw-r-- 1 liubl liubl   82 Nov  1 21:58  requirements.txt
-rw-rw-r-- 1 liubl liubl 103K Nov  1 21:58 'Skywork Community License.pdf'
-rw-rw-r-- 1 liubl liubl 260K Nov  1 21:58 'Skywork 模型社区许可协议.pdf'
-rw-rw-r-- 1 liubl liubl  411 Nov  1 21:58  special_tokens_map.json
-rw-rw-r-- 1 liubl liubl  11K Nov  1 21:58  tokenization_skywork.py
-rw-rw-r-- 1 liubl liubl  857 Nov  1 21:58  tokenizer_config.json
-rw-rw-r-- 1 liubl liubl 971K Nov  2 14:49  tokenizer.model

可以看到是这个文件没有下载完全。重新下载即可。

-rw-rw-r-- 1 liubl liubl    0 Nov  2 15:25  pytorch_model-00043-of-00053.bin