search

THUDM / GLM-4

GLM-4 series: Open Multilingual Multimodal Chat LMs | 开源多语言多模态对话模型
Apache License 2.0
5.27k stars 432 forks source link

运行trans_cli_demo.py报错 #565

Closed c2rx closed 1 month ago

c2rx commented 1 month ago

System Info / 系統信息

(ChatGLM4-9b-chat) [root@localhost basic_demo]# conda list

packages in environment at /root/anaconda3/envs/ChatGLM4-9b-chat:

#

Name Version Build Channel

_libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_gnu conda-forge about-time 4.2.1 pypi_0 pypi accelerate 0.34.2 pypi_0 pypi aiohappyeyeballs 2.4.0 pypi_0 pypi aiohttp 3.10.5 pypi_0 pypi aiosignal 1.3.1 pypi_0 pypi alive-progress 3.1.5 pypi_0 pypi async-timeout 4.0.3 pypi_0 pypi attrs 24.2.0 pypi_0 pypi autograd 1.7.0 pypi_0 pypi bzip2 1.0.8 h4bc722e_7 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge ca-certificates 2024.8.30 hbcca054_0 conda-forge certifi 2024.8.30 pypi_0 pypi charset-normalizer 3.3.2 pypi_0 pypi cma 3.2.2 pypi_0 pypi coloredlogs 15.0.1 pypi_0 pypi contourpy 1.3.0 pypi_0 pypi cycler 0.12.1 pypi_0 pypi datasets 3.0.0 pypi_0 pypi deprecated 1.2.14 pypi_0 pypi dill 0.3.8 pypi_0 pypi filelock 3.16.0 pypi_0 pypi fonttools 4.53.1 pypi_0 pypi frozenlist 1.4.1 pypi_0 pypi fsspec 2024.6.1 pypi_0 pypi grapheme 0.6.0 pypi_0 pypi huggingface-hub 0.24.7 pypi_0 pypi humanfriendly 10.0 pypi_0 pypi icu 75.1 he02047a_0 conda-forge idna 3.10 pypi_0 pypi jinja2 3.1.4 pypi_0 pypi joblib 1.4.2 pypi_0 pypi jsonschema 4.23.0 pypi_0 pypi jsonschema-specifications 2023.12.1 pypi_0 pypi jstyleson 0.0.2 pypi_0 pypi kiwisolver 1.4.7 pypi_0 pypi ld_impl_linux-64 2.40 hf3520f5_7 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libabseil 20240116.2 cxx17_he02047a_1 conda-forge libblas 3.9.0 23_linux64_openblas conda-forge libcblas 3.9.0 23_linux64_openblas conda-forge libffi 3.4.2 h7f98852_5 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libgcc 14.1.0 h77fa898_1 conda-forge libgcc-ng 14.1.0 h69a702a_1 conda-forge libgfortran 14.1.0 h69a702a_1 conda-forge libgfortran-ng 14.1.0 h69a702a_1 conda-forge libgfortran5 14.1.0 hc5f4f2c_1 conda-forge libgomp 14.1.0 h77fa898_1 conda-forge libhwloc 2.11.1 default_hecaa2ac_1000 conda-forge libiconv 1.17 hd590300_2 conda-forge liblapack 3.9.0 23_linux64_openblas conda-forge libnsl 2.0.1 hd590300_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libopenblas 0.3.27 pthreads_hac2b453_1 conda-forge libopenvino 2024.3.0 h2da1b83_0 conda-forge libopenvino-auto-batch-plugin 2024.3.0 hb045406_0 conda-forge libopenvino-auto-plugin 2024.3.0 hb045406_0 conda-forge libopenvino-dev 2024.3.0 h64d3f89_0 conda-forge libopenvino-hetero-plugin 2024.3.0 h5c03a75_0 conda-forge libopenvino-intel-cpu-plugin 2024.3.0 h2da1b83_0 conda-forge libopenvino-intel-gpu-plugin 2024.3.0 h2da1b83_0 conda-forge libopenvino-intel-npu-plugin 2024.3.0 h2da1b83_0 conda-forge libopenvino-ir-frontend 2024.3.0 h5c03a75_0 conda-forge libopenvino-onnx-frontend 2024.3.0 h07e8aee_0 conda-forge libopenvino-paddle-frontend 2024.3.0 h07e8aee_0 conda-forge libopenvino-python 2024.3.0 py310h3ba7dcc_0 conda-forge libopenvino-pytorch-frontend 2024.3.0 he02047a_0 conda-forge libopenvino-tensorflow-frontend 2024.3.0 h39126c6_0 conda-forge libopenvino-tensorflow-lite-frontend 2024.3.0 he02047a_0 conda-forge libprotobuf 4.25.3 h08a7969_0 conda-forge libsqlite 3.46.1 hadc24fc_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libstdcxx 14.1.0 hc0a3c3a_1 conda-forge libstdcxx-ng 14.1.0 h4852527_1 conda-forge libuuid 2.38.1 h0b41bf4_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge libxml2 2.12.7 he7c6b58_4 conda-forge libzlib 1.3.1 h4ab18f5_1 conda-forge markdown-it-py 3.0.0 pypi_0 pypi markupsafe 2.1.5 pypi_0 pypi matplotlib 3.9.2 pypi_0 pypi mdurl 0.1.2 pypi_0 pypi mpmath 1.3.0 pypi_0 pypi multidict 6.1.0 pypi_0 pypi multiprocess 0.70.16 pypi_0 pypi natsort 8.4.0 pypi_0 pypi ncurses 6.5 he02047a_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge networkx 3.3 pypi_0 pypi ninja 1.11.1.1 pypi_0 pypi nncf 2.12.0 pypi_0 pypi numpy 1.26.4 py310hb13e2d6_0 conda-forge nvidia-cublas-cu12 12.1.3.1 pypi_0 pypi nvidia-cuda-cupti-cu12 12.1.105 pypi_0 pypi nvidia-cuda-nvrtc-cu12 12.1.105 pypi_0 pypi nvidia-cuda-runtime-cu12 12.1.105 pypi_0 pypi nvidia-cudnn-cu12 9.1.0.70 pypi_0 pypi nvidia-cufft-cu12 11.0.2.54 pypi_0 pypi nvidia-curand-cu12 10.3.2.106 pypi_0 pypi nvidia-cusolver-cu12 11.4.5.107 pypi_0 pypi nvidia-cusparse-cu12 12.1.0.106 pypi_0 pypi nvidia-nccl-cu12 2.20.5 pypi_0 pypi nvidia-nvjitlink-cu12 12.6.68 pypi_0 pypi nvidia-nvtx-cu12 12.1.105 pypi_0 pypi ocl-icd 2.3.2 hd590300_1 conda-forge ocl-icd-system 1.0.0 1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge onnx 1.16.2 pypi_0 pypi openssl 3.3.2 hb9d3cd8_0 conda-forge openvino 2024.3.0 h85509e4_0 conda-forge openvino-telemetry 2024.1.0 pyhd8ed1ab_0 conda-forge optimum 1.22.0 pypi_0 pypi optimum-intel 1.18.0.dev0+c1ee8ac pypi_0 pypi packaging 24.1 pyhd8ed1ab_0 conda-forge pandas 2.2.2 pypi_0 pypi pillow 10.4.0 pypi_0 pypi pip 24.2 pyh8b19718_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge protobuf 5.28.1 pypi_0 pypi psutil 6.0.0 pypi_0 pypi pugixml 1.14 h59595ed_0 conda-forge pyarrow 17.0.0 pypi_0 pypi pydot 2.0.0 pypi_0 pypi pygments 2.18.0 pypi_0 pypi pymoo 0.6.1.3 pypi_0 pypi pyparsing 3.1.4 pypi_0 pypi python 3.10.0 h543edf9_3_cpython https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge python-dateutil 2.9.0.post0 pypi_0 pypi python_abi 3.10 5_cp310 conda-forge pytz 2024.2 pypi_0 pypi pyyaml 6.0.2 pypi_0 pypi readline 8.2 h8228510_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge referencing 0.35.1 pypi_0 pypi regex 2024.9.11 pypi_0 pypi requests 2.32.3 pypi_0 pypi rich 13.8.1 pypi_0 pypi rpds-py 0.20.0 pypi_0 pypi safetensors 0.4.5 pypi_0 pypi scikit-learn 1.5.2 pypi_0 pypi scipy 1.14.1 pypi_0 pypi sentencepiece 0.2.0 pypi_0 pypi setuptools 73.0.1 pyhd8ed1ab_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge six 1.16.0 pypi_0 pypi snappy 1.2.1 ha2e4443_0 conda-forge sqlite 3.46.1 h9eae976_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge sympy 1.13.2 pypi_0 pypi tabulate 0.9.0 pypi_0 pypi tbb 2021.13.0 h84d6215_0 conda-forge tbb-devel 2021.13.0 h94b29a5_0 conda-forge threadpoolctl 3.5.0 pypi_0 pypi tiktoken 0.7.0 pypi_0 pypi tk 8.6.13 noxft_h4845f30_101 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge tokenizers 0.19.1 pypi_0 pypi torch 2.4.1 pypi_0 pypi tqdm 4.66.5 pypi_0 pypi transformers 4.41.2 pypi_0 pypi triton 3.0.0 pypi_0 pypi typing-extensions 4.12.2 pypi_0 pypi tzdata 2024.1 pypi_0 pypi urllib3 2.2.3 pypi_0 pypi wheel 0.44.0 pyhd8ed1ab_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge wrapt 1.16.0 pypi_0 pypi xxhash 3.5.0 pypi_0 pypi xz 5.2.6 h166bdaf_0 conda-forge yarl 1.11.1 pypi_0 pypi

Who can help? / 谁可以帮助到您？

@zRzRzRzRzRzRzR

Information / 问题信息

Reproduction / 复现过程

最初运行的时候是报这个： GLM-4:The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results. Exception in thread Thread-2 (generate): 然后长时间无响应退出。 因为我是纯CPU环境，所以后来开始尝试使用openvino转换，也没有成功。回来再次尝试的时候，状态发生了变化，可能是安装了openvino的一些依赖造成的。求指导，谢谢！ (ChatGLM4-9b-chat) [root@localhost basic_demo]# python trans_cli_demo.py Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Loading checkpoint shards: 100%|##########################################################################################################################################################| 10/10 [00:00<00:00, 13.91it/s] Welcome to the GLM-4-9B CLI chat. Type your messages below.

You: good morning GLM-4:Exception in thread Thread-2 (generate):

Traceback (most recent call last): File "/root/anaconda3/envs/ChatGLM4-9b-chat/lib/python3.10/threading.py", line 1009, in _bootstrap_inner self.run() File "/root/anaconda3/envs/ChatGLM4-9b-chat/lib/python3.10/threading.py", line 946, in run self._target(*self._args, *self._kwargs) File "/root/anaconda3/envs/ChatGLM4-9b-chat/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(args, **kwargs) File "/root/anaconda3/envs/ChatGLM4-9b-chat/lib/python3.10/site-packages/transformers/generation/utils.py", line 1758, in generate result = self._sample( File "/root/anaconda3/envs/ChatGLM4-9b-chat/lib/python3.10/site-packages/transformers/generation/utils.py", line 2449, in _sample model_kwargs = self._update_model_kwargs_for_generation( File "/root/.cache/huggingface/modules/transformers_modules/glm-4-9b-chat/modeling_chatglm.py", line 929, in _update_model_kwargs_for_generation cache_name, cache = self._extract_past_from_model_output(outputs) ValueError: too many values to unpack (expected 2) Traceback (most recent call last): File "/root/Glm4/GLM-4/basic_demo/trans_cli_demo.py", line 112, in for new_token in streamer: File "/root/anaconda3/envs/ChatGLM4-9b-chat/lib/python3.10/site-packages/transformers/generation/streamers.py", line 223, in next value = self.text_queue.get(timeout=self.timeout) File "/root/anaconda3/envs/ChatGLM4-9b-chat/lib/python3.10/queue.py", line 179, in get raise Empty _queue.Empty

Expected behavior / 期待表现

能正常运行就行，最好能给出纯CPU环境运行的一些建议，谢谢！！

ghost commented 1 month ago

https://github.com/THUDM/GLM-4/blob/main/basic_demo/trans_batch_demo.py 请在代码第51行修改为：device_map="cpu"，94行修改为timeout=600，即可正常运行推理脚本 cpu上推理耗时会久一点