Closed rinne1998 closed 1 year ago
谢谢反馈,这个是windows安装相关的问题,我们正在fix,您可以先试试在wsl下运行。修复好后我们将通知您。
请将jittor更新至1.3.7.3, jtorch更新到0.1.3, jittorllms更新到最新可修复该问题
修复工作辛苦!我在更新jittor和jtorch之后运行python.exe cli_demo.py chatglm
,无法加载模型,完整提示:
PS D:\Code\JittorLLMs> C:\Users\parsl\AppData\Local\Programs\Python\Python38\python.exe cli_demo.py chatglm
[i 0404 04:06:40.566000 04 compiler.py:955] Jittor(1.3.7.3) src: c:\users\parsl\appdata\local\programs\python\python38\lib\site-packages\jittor
[i 0404 04:06:40.610000 04 compiler.py:956] cl at C:\Users\parsl\.cache\jittor\msvc\VC\_\_\_\_\_\bin\cl.exe(19.29.30133)
[i 0404 04:06:40.610000 04 compiler.py:957] cache_path: C:\Users\parsl\.cache\jittor\jt1.3.7\cl\py3.8.10\Windows-10-10.x0f\IntelRCoreTMi5xbf\default
[i 0404 04:06:40.613000 04 install_cuda.py:93] cuda_driver_version: [11, 6, 0]
[i 0404 04:06:40.648000 04 __init__.py:411] Found C:\Users\parsl\.cache\jittor\jtcuda\cuda11.2_cudnn8_win\bin\nvcc.exe(11.2.67) at C:\Users\parsl\.cache\jittor\jtcuda\cuda11.2_cudnn8_win\bin\nvcc.exe.
[i 0404 04:06:40.733000 04 compiler.py:1010] cuda key:cu11.2.67
[i 0404 04:06:40.735000 04 __init__.py:227] Total mem: 39.85GB, using 13 procs for compiling.
Compiling jit_utils_core(5/5) used: 2.145s eta: 0.000s
[i 0404 04:06:43.681000 04 jit_compiler.cc:28] Load cc_path: C:\Users\parsl\.cache\jittor\msvc\VC\_\_\_\_\_\bin\cl.exe
[i 0404 04:06:43.682000 04 init.cc:62] Found cuda archs: [61,]
[i 0404 04:06:44.689000 04 compile_extern.py:522] mpicc not found, distribution disabled.
[w 0404 04:06:44.737000 04 compile_extern.py:203] CUDA related path found in LD_LIBRARY_PATH or PATH(['', 'C', '\\Users\\parsl\\.cache\\jittor\\jtcuda\\cuda11.2_cudnn8_win\\lib64', '', 'C', '\\Users\\parsl\\.cache\\jittor\\mkl\\dnnl_win_2.2.0_cpu_vcomp\\bin', '', 'C', '\\Users\\parsl\\.cache\\jittor\\mkl\\dnnl_win_2.2.0_cpu_vcomp\\lib', '', 'C', '\\Users\\parsl\\.cache\\jittor\\jt1.3.7\\cl\\py3.8.10\\Windows-10-10.x0f\\IntelRCoreTMi5xbf\\default', '', 'C', '\\Users\\parsl\\.cache\\jittor\\jt1.3.7\\cl\\py3.8.10\\Windows-10-10.x0f\\IntelRCoreTMi5xbf\\default\\cu11.2.67', '', 'C', '\\Users\\parsl\\.cache\\jittor\\jtcuda\\cuda11.2_cudnn8_win\\bin', '', 'C', '\\Users\\parsl\\.cache\\jittor\\jtcuda\\cuda11.2_cudnn8_win\\lib\\x64', '', 'C', '\\Users\\parsl\\.cache\\jittor\\msvc\\win10_kits\\lib\\ucrt\\x64', '', 'C', '\\Users\\parsl\\.cache\\jittor\\msvc\\win10_kits\\lib\\um\\x64', '', 'C', '\\Users\\parsl\\.cache\\jittor\\msvc\\VC\\lib', '', 'c', '\\users\\parsl\\appdata\\local\\programs\\python\\python38\\libs', 'C', '\\Users\\parsl\\.cache\\jittor\\msvc\\VC\\_\\_\\_\\_\\_\\bin', 'C', '\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.6\\bin', 'C', '\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.6\\libnvvp', 'C', '\\Program Files\\Eclipse Adoptium\\jdk-8.0.362.9-hotspot\\bin', 'C', '\\Windows\\system32', 'C', '\\Windows', 'C', '\\Windows\\System32\\Wbem', 'C', '\\Windows\\System32\\WindowsPowerShell\\v1.0\\', 'C', '\\Windows\\System32\\OpenSSH\\', 'C', '\\Program Files\\dotnet\\', 'C', '\\Program Files (x86)\\NVIDIA Corporation\\PhysX\\Common', 'C', '\\Program Files (x86)\\NetSarang\\Xshell 7\\', 'C', '\\Program Files (x86)\\NetSarang\\Xftp 7\\', 'C', '\\Program Files\\nodejs\\', 'C', '\\Program Files\\Git\\cmd', 'C', '\\Program Files (x86)\\HP\\Common\\HPDestPlgIn\\', 'C', '\\Program Files\\IBM\\SPSS\\Statistics\\25\\JRE\\bin', 'C', '\\Program Files\\NVIDIA Corporation\\Nsight Compute 2022.1.0\\', 'C', '\\Users\\parsl\\AppData\\Local\\Programs\\Python\\Python37\\Scripts\\', 'C', '\\Users\\parsl\\AppData\\Local\\Programs\\Python\\Python37\\', 'C', '\\Users\\parsl\\AppData\\Local\\Microsoft\\WindowsApps', 'C', '\\Users\\parsl\\AppData\\Local\\Programs\\Microsoft VS Code\\bin', 'C', '\\Users\\parsl\\AppData\\Roaming\\npm', 'C', '\\Users\\parsl\\.dotnet\\tools', 'C', '\\Users\\parsl\\AppData\\Local\\Programs\\Fiddler']), This path may cause jittor
found the wrong libs, please unset LD_LIBRARY_PATH and remove cuda lib path in Path.
Or you can let jittor install cuda for you: `python3.x -m jittor_utils.install_cuda`
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Loading checkpoint shards: 0%| | 0/8 [00:00<?, ?it/s]
Traceback (most recent call last):
File "C:\Users\parsl\AppData\Local\Programs\Python\Python38\lib\site-packages\transformers\modeling_utils.py", line 415, in load_state_dict
return torch.load(checkpoint_file, map_location="cpu")
File "C:\Users\parsl\AppData\Local\Programs\Python\Python38\lib\site-packages\jtorch\__init__.py", line 228, in load
return jt.load(path)
File "C:\Users\parsl\AppData\Local\Programs\Python\Python38\lib\site-packages\jittor\__init__.py", line 1124, in load
model_dict = safeunpickle(path)
File "C:\Users\parsl\AppData\Local\Programs\Python\Python38\lib\site-packages\jittor\__init__.py", line 98, in safeunpickle
model_dict = load_pytorch(path)
File "C:\Users\parsl\AppData\Local\Programs\Python\Python38\lib\site-packages\jittor_utils\load_pytorch.py", line 220, in load_pytorch
result = unpickler.load()
File "C:\Users\parsl\AppData\Local\Programs\Python\Python38\lib\site-packages\jittor_utils\load_pytorch.py", line 43, in persistent_load
load_tensor(contents, dtype, nbytes, key, _maybe_decode_ascii(location))
File "C:\Users\parsl\AppData\Local\Programs\Python\Python38\lib\site-packages\jittor_utils\load_pytorch.py", line 20, in load_tensor
loaded_storages[key] = contents.read_var(name, dtype)
RuntimeError: Wrong inputs arguments, Please refer to examples(help(jt.ZipFile.read_var)).
Types of your inputs are:
self = ZipFile,
args = (str, type, ),
The function declarations are:
inline VarHolder* read_var(const string& filename, NanoString dtype=ns_uint8)
Failed reason:[f 0404 04:06:52.096000 04 miniz.h:1365] Check failed: mz_zip_reader_file_stat(zip_archive.get(), key, &stat)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\parsl\AppData\Local\Programs\Python\Python38\lib\site-packages\transformers\modeling_utils.py", line 419, in load_state_dict
if f.read(7) == "version":
UnicodeDecodeError: 'gbk' codec can't decode byte 0x80 in position 64: illegal multibyte sequence
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "cli_demo.py", line 8, in <module>
model = models.get_model(args)
File "D:\Code\JittorLLMs\models\__init__.py", line 42, in get_model
return module.get_model(args)
File "D:\Code\JittorLLMs\models\chatglm\__init__.py", line 48, in get_model
return ChatGLMMdoel(args)
File "D:\Code\JittorLLMs\models\chatglm\__init__.py", line 22, in __init__
self.model = AutoModel.from_pretrained(os.path.dirname(__file__), trust_remote_code=True)
File "C:\Users\parsl\AppData\Local\Programs\Python\Python38\lib\site-packages\transformers\models\auto\auto_factory.py", line 459, in from_pretrained
return model_class.from_pretrained(
File "C:\Users\parsl\AppData\Local\Programs\Python\Python38\lib\site-packages\transformers\modeling_utils.py", line 2478, in from_pretrained
) = cls._load_pretrained_model(
File "C:\Users\parsl\AppData\Local\Programs\Python\Python38\lib\site-packages\transformers\modeling_utils.py", line 2780, in _load_pretrained_model
state_dict = load_state_dict(shard_file)
File "C:\Users\parsl\AppData\Local\Programs\Python\Python38\lib\site-packages\transformers\modeling_utils.py", line 431, in load_state_dict
raise OSError(
OSError: Unable to load weights from pytorch checkpoint file for 'D:\Code\JittorLLMs\models\chatglm\pytorch_model-00001-of-00008.bin' at 'D:\Code\JittorLLMs\models\chatglm\pytorch_model-00001-of-00008.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
尝试过重新编译jittor、重新下载pytorch_model-00001-of-00008.bin
、更换transformers(4.26.1、4.27.4)版本,但问题依旧,测试使用的pytorch_model-00001-of-00008.bin
信息:
名称: pytorch_model-00001-of-00008.bin
大小: 1904491802 字节 (1816 MiB)
CRC32: 27214D29
CRC64: 80E50407F55BF305
SHA256: fe5bac6bfa5b5404ddfe3fabe04862b785e013afd7b308b7beca08239f9489fa
SHA1: 091be970fb9196b4910fd04629418839346ae5c6
BLAKE2sp: d38ca44ad13a638096f5efb9ea5a416f9f89cab174b12eabab6176f98758b4d6
该权重可以在非jittor环境的transformers中成功加载使用。
E:\JittorLLMs>cli_demo.py chatglm
[i 0404 11:11:43.180000 52 compiler.py:955] Jittor(1.3.7.3) src: e:\users\administrator\appdata\local\programs\python\python310\lib\site-packages\jittor
[i 0404 11:11:43.268000 52 compiler.py:956] cl at C:\Users\Administrator\.cache\jittor\msvc\VC\_\_\_\_\_\bin\cl.exe(19.29.30133)
[i 0404 11:11:43.269000 52 compiler.py:957] cache_path: C:\Users\Administrator\.cache\jittor\jt1.3.7\cl\py3.10.10\Windows-10-10.x91\IntelRXeonRCPUx6b\default
[i 0404 11:11:43.311000 52 __init__.py:227] Total mem: 15.93GB, using 5 procs for compiling.
[i 0404 11:11:44.436000 52 jit_compiler.cc:28] Load cc_path: C:\Users\Administrator\.cache\jittor\msvc\VC\_\_\_\_\_\bin\cl.exe
[i 0404 11:11:44.607000 52 compile_extern.py:522] mpicc not found, distribution disabled.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Loading checkpoint shards: 0%| | 0/8 [00:00<?, ?it/s]
Traceback (most recent call last):
File "E:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\modeling_utils.py", line 415, in load_state_dict
return torch.load(checkpoint_file, map_location="cpu")
File "E:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\jtorch\__init__.py", line 228, in load
return jt.load(path)
File "E:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\jittor\__init__.py", line 1124, in load
model_dict = safeunpickle(path)
File "E:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\jittor\__init__.py", line 98, in safeunpickle
model_dict = load_pytorch(path)
File "E:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\jittor_utils\load_pytorch.py", line 220, in load_pytorch
result = unpickler.load()
File "E:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\jittor_utils\load_pytorch.py", line 43, in persistent_load
load_tensor(contents, dtype, nbytes, key, _maybe_decode_ascii(location))
File "E:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\jittor_utils\load_pytorch.py", line 20, in load_tensor
loaded_storages[key] = contents.read_var(name, dtype)
RuntimeError: Wrong inputs arguments, Please refer to examples(help(jt.ZipFile.read_var)).
Types of your inputs are:
self = ZipFile,
args = (str, type, ),
The function declarations are:
inline VarHolder* read_var(const string& filename, NanoString dtype=ns_uint8)
Failed reason:[f 0404 11:11:48.591000 52 miniz.h:1365] Check failed: mz_zip_reader_file_stat(zip_archive.get(), key, &stat)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "E:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\modeling_utils.py", line 419, in load_state_dict
if f.read(7) == "version":
UnicodeDecodeError: 'gbk' codec can't decode byte 0x80 in position 64: illegal multibyte sequence
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "E:\JittorLLMs\cli_demo.py", line 8, in <module>
model = models.get_model(args)
File "E:\JittorLLMs\models\__init__.py", line 42, in get_model
return module.get_model(args)
File "E:\JittorLLMs\models\chatglm\__init__.py", line 48, in get_model
return ChatGLMMdoel(args)
File "E:\JittorLLMs\models\chatglm\__init__.py", line 22, in __init__
self.model = AutoModel.from_pretrained(os.path.dirname(__file__), trust_remote_code=True)
File "E:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\models\auto\auto_factory.py", line 466, in from_pretrained
return model_class.from_pretrained(
File "E:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\modeling_utils.py", line 2646, in from_pretrained
) = cls._load_pretrained_model(
File "E:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\modeling_utils.py", line 2955, in _load_pretrained_model
state_dict = load_state_dict(shard_file)
File "E:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\modeling_utils.py", line 431, in load_state_dict
raise OSError(
OSError: Unable to load weights from pytorch checkpoint file for 'E:\JittorLLMs\models\chatglm\pytorch_model-00001-of-00008.bin' at 'E:\JittorLLMs\models\chatglm\pytorch_model-00001-of-00008.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
我的也是一样情况
谢谢您的反馈,jittor更新到1.3.7.4即可
感谢排障!已成功运行 @bearworks
你好,在尝试使用ChatGLM:
python.exe cli_demo.py chatglm
出现以下错误:
执行
python -m jittor_utils.install_cuda
结果:如果单独
import jtorch
,也会出现上述的报错。环境:
已安装的依赖:
多次尝试重新compile问题依旧。