search

THUDM / ChatGLM-6B

ChatGLM-6B: An Open Bilingual Dialogue Language Model | 开源双语对话语言模型
Apache License 2.0
40.47k stars 5.19k forks source link

[BUG/Help] cli_demo.py/web_demo.py 启动没报错，但没返回 #1414

Open luo-ky opened 11 months ago

luo-ky commented 11 months ago

Is there an existing issue for this?

Current Behavior

cli_demo.py/api.py 启动没报错，但没返回，用的 ChatGLM2-6B-int4 模型，就一个bin的那个

就一直这样：

欢迎使用 ChatGLM2-6B 模型，输入内容即可进行对话，clear 清空对话历史，stop 终止程序

用户：你好

ChatGLM-6B：

下面是配置信息 ，系统显存是10G

Expected Behavior

aiofiles 23.2.1 pypi_0 pypi altair 5.1.2 pypi_0 pypi annotated-types 0.6.0 pypi_0 pypi anyio 3.7.1 pypi_0 pypi argon2-cffi 21.1.0 pypi_0 pypi attrs 21.2.0 pypi_0 pypi babel 2.9.1 pypi_0 pypi backcall 0.2.0 pypi_0 pypi bleach 4.1.0 pypi_0 pypi brotlipy 0.7.0 py38h27cfd23_1003 defaults ca-certificates 2021.7.5 h06a4308_1 defaults cachetools 4.2.4 pypi_0 pypi certifi 2021.5.30 py38h06a4308_0 defaults cffi 1.14.6 py38h400218f_0 defaults chardet 4.0.0 py38h06a4308_1003 defaults click 8.1.7 pypi_0 pypi conda 4.10.3 py38h06a4308_0 defaults conda-package-handling 1.7.3 py38h27cfd23_1 defaults cpm-kernels 1.0.11 pypi_0 pypi cryptography 3.4.7 py38hd23ed53_0 defaults cycler 0.11.0 pypi_0 pypi debugpy 1.5.1 pypi_0 pypi decorator 5.1.0 pypi_0 pypi defusedxml 0.7.1 pypi_0 pypi entrypoints 0.3 pypi_0 pypi exceptiongroup 1.1.3 pypi_0 pypi fastapi 0.104.0 pypi_0 pypi ffmpy 0.3.1 pypi_0 pypi filelock 3.12.4 pypi_0 pypi fonttools 4.28.2 pypi_0 pypi fsspec 2023.9.2 pypi_0 pypi google-auth 2.3.3 pypi_0 pypi google-auth-oauthlib 0.4.6 pypi_0 pypi gradio 3.44.0 pypi_0 pypi gradio-client 0.5.0 pypi_0 pypi grpcio 1.42.0 pypi_0 pypi h11 0.14.0 pypi_0 pypi httpcore 0.18.0 pypi_0 pypi httpx 0.25.0 pypi_0 pypi huggingface-hub 0.18.0 pypi_0 pypi idna 2.10 pyhd3eb1b0_0 defaults importlib-metadata 4.8.2 pypi_0 pypi importlib-resources 5.4.0 pypi_0 pypi ipykernel 6.5.1 pypi_0 pypi ipython 7.29.0 pypi_0 pypi ipython-genutils 0.2.0 pypi_0 pypi ipywidgets 7.6.5 pypi_0 pypi jedi 0.18.1 pypi_0 pypi jinja2 3.0.3 pypi_0 pypi json5 0.9.6 pypi_0 pypi jsonschema 4.2.1 pypi_0 pypi jupyter-client 7.1.0 pypi_0 pypi jupyter-core 4.9.1 pypi_0 pypi jupyter-server 1.12.0 pypi_0 pypi jupyterlab 3.2.4 pypi_0 pypi jupyterlab-language-pack-zh-cn 3.2.post2 pypi_0 pypi jupyterlab-pygments 0.1.2 pypi_0 pypi jupyterlab-server 2.8.2 pypi_0 pypi jupyterlab-widgets 1.0.2 pypi_0 pypi kiwisolver 1.3.2 pypi_0 pypi latex2mathml 3.76.0 pypi_0 pypi ld_impl_linux-64 2.35.1 h7274673_9 defaults libffi 3.3 he6710b0_2 defaults libgcc-ng 9.3.0 h5101ec6_17 defaults libgomp 9.3.0 h5101ec6_17 defaults libstdcxx-ng 9.3.0 hd4cf53a_17 defaults markdown 3.3.6 pypi_0 pypi markupsafe 2.0.1 pypi_0 pypi matplotlib 3.5.0 pypi_0 pypi matplotlib-inline 0.1.3 pypi_0 pypi mdtex2html 1.2.0 pypi_0 pypi mistune 0.8.4 pypi_0 pypi nbclassic 0.3.4 pypi_0 pypi nbclient 0.5.9 pypi_0 pypi nbconvert 6.3.0 pypi_0 pypi nbformat 5.1.3 pypi_0 pypi ncurses 6.2 he6710b0_1 defaults nest-asyncio 1.5.1 pypi_0 pypi notebook 6.4.6 pypi_0 pypi numpy 1.21.4 pypi_0 pypi oauthlib 3.1.1 pypi_0 pypi openssl 1.1.1k h27cfd23_0 defaults orjson 3.9.9 pypi_0 pypi packaging 21.3 pypi_0 pypi pandas 2.0.3 pypi_0 pypi pandocfilters 1.5.0 pypi_0 pypi parso 0.8.2 pypi_0 pypi pexpect 4.8.0 pypi_0 pypi pickleshare 0.7.5 pypi_0 pypi pillow 8.4.0 pypi_0 pypi pip 21.1.3 py38h06a4308_0 defaults prometheus-client 0.12.0 pypi_0 pypi prompt-toolkit 3.0.22 pypi_0 pypi protobuf 3.19.1 pypi_0 pypi psutil 5.9.6 pypi_0 pypi ptyprocess 0.7.0 pypi_0 pypi pyasn1 0.4.8 pypi_0 pypi pyasn1-modules 0.2.8 pypi_0 pypi pycosat 0.6.3 py38h7b6447c_1 defaults pycparser 2.20 py_2 defaults pydantic 2.4.2 pypi_0 pypi pydantic-core 2.10.1 pypi_0 pypi pydub 0.25.1 pypi_0 pypi pygments 2.10.0 pypi_0 pypi pyopenssl 20.0.1 pyhd3eb1b0_1 defaults pyparsing 3.0.6 pypi_0 pypi pyrsistent 0.18.0 pypi_0 pypi pysocks 1.7.1 py38h06a4308_0 defaults python 3.8.10 h12debd9_8 defaults python-dateutil 2.8.2 pypi_0 pypi python-multipart 0.0.6 pypi_0 pypi pytz 2021.3 pypi_0 pypi pyyaml 6.0.1 pypi_0 pypi pyzmq 22.3.0 pypi_0 pypi readline 8.1 h27cfd23_0 defaults regex 2023.10.3 pypi_0 pypi requests 2.25.1 pyhd3eb1b0_0 defaults requests-oauthlib 1.3.0 pypi_0 pypi rsa 4.8 pypi_0 pypi ruamel_yaml 0.15.100 py38h27cfd23_0 defaults safetensors 0.4.0 pypi_0 pypi semantic-version 2.10.0 pypi_0 pypi send2trash 1.8.0 pypi_0 pypi sentencepiece 0.1.99 pypi_0 pypi setuptools 52.0.0 py38h06a4308_0 defaults setuptools-scm 6.3.2 pypi_0 pypi six 1.16.0 pyhd3eb1b0_0 defaults sniffio 1.2.0 pypi_0 pypi sqlite 3.36.0 hc218d9a_0 defaults starlette 0.27.0 pypi_0 pypi supervisor 4.2.2 pypi_0 pypi tensorboard 2.7.0 pypi_0 pypi tensorboard-data-server 0.6.1 pypi_0 pypi tensorboard-plugin-wit 1.8.0 pypi_0 pypi terminado 0.12.1 pypi_0 pypi testpath 0.5.0 pypi_0 pypi tk 8.6.10 hbc83047_0 defaults tokenizers 0.13.3 pypi_0 pypi tomli 1.2.2 pypi_0 pypi toolz 0.12.0 pypi_0 pypi torch 1.10.0+cu113 pypi_0 pypi torchvision 0.11.1+cu113 pypi_0 pypi tornado 6.1 pypi_0 pypi tqdm 4.61.2 pyhd3eb1b0_1 defaults traitlets 5.1.1 pypi_0 pypi transformers 4.30.2 pypi_0 pypi typing-extensions 4.8.0 pypi_0 pypi tzdata 2023.3 pypi_0 pypi urllib3 1.26.6 pyhd3eb1b0_1 defaults uvicorn 0.23.2 pypi_0 pypi wcwidth 0.2.5 pypi_0 pypi webencodings 0.5.1 pypi_0 pypi websocket-client 1.2.1 pypi_0 pypi websockets 11.0.3 pypi_0 pypi werkzeug 2.0.2 pypi_0 pypi wheel 0.36.2 pyhd3eb1b0_0 defaults widgetsnbextension 3.5.2 pypi_0 pypi xz 5.2.5 h7b6447c_0 defaults yaml 0.2.5 h7b6447c_0 defaults zipp 3.6.0 pypi_0 pypi zlib 1.2.11 h7b6447c_3 defaults

Steps To Reproduce

启动日志（下面启动时打印的警告信息）：： Some weights of the model checkpoint at THUDM/chatglm2-6b-int4 were not used when initializing ChatGLMForConditionalGeneration: ['transformer.layers.0.mlp.dense_h_to_4h.weight', 'transformer.layers.20.mlp.dense_4h_to_h.weight', 'transformer.layers.14.attention.query_key_value.weight_scale', 'transformer.layers.15.mlp.dense_h_to_4h.bias'.................

This IS expected if you are initializing ChatGLMForConditionalGeneration from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model)....

You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. INFO: Started server process [5996] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:6006 (Press CTRL+C to quit)

Environment

- OS:Ubuntu 
- Python:3.8
- Transformers: 4.30
- PyTorch: 3.10
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) :true

Anything else?

No response

zhaoxiongjun commented 11 months ago

+1 ， 用的是chatglm-6b的参数

mularo commented 11 months ago

+1,运行web_demo.py也没有输出，运行web_demo2.py有输出，但是一直输出这个看不懂的字符

JiXia830 commented 10 months ago

+1，web_demo正常启动，但是一提问就卡住

794458394 commented 10 months ago

解决了吗哥，我也是这样的问题