search

THUDM / GLM-4

GLM-4 series: Open Multilingual Multimodal Chat LMs | 开源多语言多模态对话模型
Apache License 2.0
4.74k stars 385 forks source link

运行官方的示例代码，乱输出 #243

Closed Barbery closed 3 months ago

Barbery commented 3 months ago

System Info / 系統信息

代码是readme中的示例代码，具体如下：

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

device = "cuda"

tokenizer = AutoTokenizer.from_pretrained("THUDM/glm-4-9b-chat", trust_remote_code=True)

query = "你好"

inputs = tokenizer.apply_chat_template([{"role": "user", "content": query}],
                                       add_generation_prompt=True,
                                       tokenize=True,
                                       return_tensors="pt",
                                       return_dict=True
                                       )

inputs = inputs.to(device)
model = AutoModelForCausalLM.from_pretrained(
    "THUDM/glm-4-9b-chat",
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
    trust_remote_code=True
).to(device).eval()

gen_kwargs = {"max_length": 2500, "do_sample": True, "top_k": 1}
with torch.no_grad():
    outputs = model.generate(**inputs, **gen_kwargs)
    outputs = outputs[:, inputs['input_ids'].shape[1]:]
    print(tokenizer.decode(outputs[0], skip_special_tokens=True))

输出结果：

Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 17.47it/s]
/to/to$ý之於於於愈愈愈愈愈愈愈愈愈愈愈愈愈起起起起起起起起起起起起起起起起起起起起起起起起来起来起来起来起来起来起来起来起来起来起来起来起来起来起来起来起来起来起来起来起来乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎乎之之之之之之之之之之之之之之之之之之之之之之之之之之之之之之之之之之之之之之之既既之之之之之之之之之之之之之乎之乎之之之之之之之之之之之之之之之既之之之之之之之之之之之之既之下之下既既既既既既既既既既既既既既既既既既既既既既既既既既既既既既既既既既既既既既既既既既之下之下之下之下之下之下之下之下之下之下既既之下之下之下之下之下之下之下之下之下之下既既既既既之下之下之下既之下之下之下之下之下之下之下既既既既既既既既既既既之下之下之下之下之下既既之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下之下

Who can help? / 谁可以帮助到您？

No response

Information / 问题信息

Reproduction / 复现过程

运行上述脚本

Expected behavior / 期待表现

期待正常回复

zRzRzRzRzRzRzR commented 3 months ago

check你的显卡和驱动

  1. 最好使用BF16加载并确定你的显卡驱动是否大于535版本，cuda11.8或者12以上，另外，可以给我一下你的环境
Barbery commented 3 months ago

@zRzRzRzRzRzRzR

  1. 示例脚本里有指定torch_dtype=torch.bfloat16,这个，哪里还需要改动吗？
  2. 版本信息：NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 |
zRzRzRzRzRzRzR commented 3 months ago

那看上去这个环境是没有问题的，卡是支持bloat16的吧。给个环境我在check一下

Barbery commented 3 months ago

@zRzRzRzRzRzRzR 显卡是3090 应该支持bfloat16的。你说的给个环境是具体还需要提供什么信息？我没搞明白，抱歉。

zRzRzRzRzRzRzR commented 3 months ago

就是python包的环境，现在显卡和驱动看上去没有问题了

Barbery commented 3 months ago

@zRzRzRzRzRzRzR 

# pip list
Package                      Version
---------------------------- ---------------------
absl-py                      1.4.0
accelerate                   0.26.1
aiofiles                     23.1.0
aiohttp                      3.9.2
aiosignal                    1.3.1
altair                       4.2.2
annotated-types              0.6.0
anyio                        4.2.0
appdirs                      1.4.4
appnope                      0.1.3
argon2-cffi                  23.1.0
argon2-cffi-bindings         21.2.0
arxiv                        2.1.0
asgiref                      3.7.2
asrt-sdk                     1.2.0
astor                        0.8.1
asttokens                    2.4.1
astunparse                   1.6.3
async-timeout                4.0.3
async-tio                    1.3.2
attrs                        23.2.0
bcrypt                       4.0.1
beautifulsoup4               4.12.2
blinker                      1.7.0
boltons                      23.0.0
brotlipy                     0.7.0
bs4                          0.0.1
cached-property              1.5.2
cachetools                   5.3.1
certifi                      2023.11.17
cffi                         1.16.0
chardet                      5.1.0
charset-normalizer           3.3.2
chatgptpy                    1.0.8
chex                         0.1.85
click                        8.1.7
cmake                        3.26.3
colorama                     0.4.6
colorcet                     3.1.0
comm                         0.2.1
conda                        23.3.1
conda-content-trust          0.1.3
conda-package-handling       2.0.2
conda_package_streaming      0.7.0
contourpy                    1.0.7
cpm-kernels                  1.0.11
cryptography                 39.0.1
cssselect2                   0.7.0
curl-cffi                    0.5.6
cycler                       0.11.0
Cython                       0.29.35
dataclasses-json             0.6.3
datasets                     2.12.0
DateTime                     5.4
debugpy                      1.8.0
decorator                    5.1.1
derpconf                     0.8.3
dill                         0.3.6
distro                       1.9.0
Django                       4.1.9
django-crontab               0.7.1
django-environ               0.10.0
dlib                         19.24.2
docker                       6.1.2
docker-compose               1.29.2
docker-pycreds               0.4.0
dockerpty                    0.4.1
docopt                       0.6.2
docx2txt                     0.8
easy_install                 66.0.2
einops                       0.8.0
entrypoints                  0.4
environs                     9.5.0
etils                        1.6.0
exceptiongroup               1.2.0
executing                    2.0.1
ez_setup                     0.9
face-recognition             1.3.0
face-recognition-models      0.3.0
fastapi                      0.109.0
feedparser                   6.0.10
ffmpeg-python                0.2.0
ffmpy                        0.3.0
filelock                     3.13.1
flash-attn                   2.5.9.post1
Flask                        3.0.2
flatbuffers                  23.5.26
flax                         0.6.4
fonttools                    4.39.4
frozenlist                   1.4.1
fsspec                       2023.12.2
future                       0.18.3
gast                         0.4.0
gensim                       4.3.1
gitdb                        4.0.10
GitPython                    3.1.31
google-auth                  2.19.0
google-auth-oauthlib         0.4.6
google-pasta                 0.2.0
googleapis-common-protos     1.59.0
gradio                       4.14.0
gradio_client                0.8.0
greenlet                     3.0.3
grpcio                       1.58.0
grpcio-tools                 1.54.2
h11                          0.14.0
h5py                         3.8.0
httpcore                     1.0.4
httplib2                     0.22.0
httpx                        0.27.0
huggingface-hub              0.23.4
idna                         3.6
imageio                      2.31.0
imageio-ffmpeg               0.4.8
importlib-metadata           7.0.1
importlib-resources          6.1.1
ipykernel                    6.29.0
ipython                      8.18.1
itsdangerous                 2.1.2
jax                          0.4.23
jaxlib                       0.4.23+cuda12.cudnn89
jedi                         0.19.1
jieba                        0.42.1
Jinja2                       3.1.3
joblib                       1.3.2
jsonpatch                    1.33
jsonpointer                  2.4
jsonschema                   3.2.0
jupyter_client               8.6.0
jupyter_core                 5.7.1
keras                        2.8.0
Keras-Preprocessing          1.1.2
kiwisolver                   1.4.4
langchain                    0.1.1
langchain-community          0.0.13
langchain-core               0.1.12
langchainhub                 0.1.14
langsmith                    0.0.82
latex2mathml                 3.77.0
libclang                     16.0.0
libthumbor                   2.0.2
linkify-it-py                2.0.2
lit                          16.0.5
llvmlite                     0.40.0
loguru                       0.7.2
lxml                         4.9.3
Markdown                     3.4.3
markdown-it-py               2.2.0
MarkupSafe                   2.1.4
marshmallow                  3.20.2
matplotlib                   3.7.1
matplotlib-inline            0.1.6
mdit-py-plugins              0.3.3
mdtex2html                   1.2.0
mdurl                        0.1.2
milvus                       2.3.5
minio                        7.2.3
mkl-fft                      1.3.6
mkl-random                   1.2.2
mkl-service                  2.4.0
ml-dtypes                    0.3.2
more-itertools               9.1.0
moviepy                      1.0.3
mpmath                       1.3.0
msgpack                      1.0.7
multidict                    6.0.4
multiprocess                 0.70.14
mypy-extensions              1.0.0
mysql                        0.0.3
mysql-connector-python       8.1.0
mysqlclient                  2.2.0
nest-asyncio                 1.6.0
networkx                     3.2.1
nltk                         3.8.1
numba                        0.57.0
numpy                        1.26.3
nvidia-cublas-cu12           12.1.3.1
nvidia-cuda-cupti-cu12       12.1.105
nvidia-cuda-nvcc-cu12        12.4.131
nvidia-cuda-nvrtc-cu12       12.1.105
nvidia-cuda-runtime-cu12     12.1.105
nvidia-cudnn-cu12            8.9.2.26
nvidia-cufft-cu12            11.0.2.54
nvidia-curand-cu12           10.3.2.106
nvidia-cusolver-cu12         11.4.5.107
nvidia-cusparse-cu12         12.1.0.106
nvidia-nccl-cu12             2.18.1
nvidia-nvjitlink-cu12        12.4.127
nvidia-nvtx-cu12             12.1.105
oauthlib                     3.2.2
openai                       1.13.3
openai-whisper               20230314
OpenAIAuth                   3.0.0
opencv-python                4.7.0.72
opencv-python-headless       4.7.0.72
opt-einsum                   3.3.0
optax                        0.1.7
orbax                        0.1.9
orbax-checkpoint             0.4.8
orjson                       3.9.15
packaging                    23.2
paddle-bfloat                0.1.2
paddlepaddle-gpu             2.3.0
pandas                       2.2.0
paramiko                     3.1.0
parrots                      0.1.7
parso                        0.8.3
pathtools                    0.1.2
peft                         0.3.0
pexpect                      4.9.0
piexif                       1.1.3
pillow                       10.2.0
pip                          24.1
platformdirs                 4.1.0
pluggy                       1.0.0
pocketsphinx                 5.0.1
proglog                      0.1.10
progressbar                  2.5
prompt-toolkit               3.0.43
protobuf                     4.25.2
psutil                       5.9.8
ptyprocess                   0.7.0
pure-eval                    0.2.2
pyarrow                      15.0.0
pyasn1                       0.5.0
pyasn1-modules               0.3.0
pycosat                      0.6.4
pycparser                    2.21
pycryptodome                 3.20.0
pycurl                       7.45.2
pydantic                     2.5.3
pydantic_core                2.14.6
pydash                       7.0.4
pydeck                       0.8.1b0
pydub                        0.25.1
Pygments                     2.17.2
PyJWT                        2.8.0
pymilvus                     2.3.5
Pympler                      1.0.1
PyMySQL                      1.1.0
PyNaCl                       1.5.0
pyOpenSSL                    23.0.0
pyparsing                    3.0.9
pypinyin                     0.49.0
pyrsistent                   0.19.3
PySocks                      1.7.1
pysrt                        1.1.2
python-dateutil              2.8.2
python-dotenv                1.0.1
python-multipart             0.0.6
python-speech-features       0.6
python-utils                 3.6.0
pytorch-triton               2.1.0+440fd1bf20
pytz                         2023.3.post1
pytz-deprecation-shim        0.1.0.post0
PyYAML                       6.0.1
pyzmq                        25.1.2
regex                        2023.12.25
reportlab                    4.0.4
requests                     2.31.0
requests-oauthlib            1.3.1
responses                    0.18.0
result                       0.16.1
revChatGPT                   6.8.6
rich                         13.4.1
rouge                        1.0.1
rouge-chinese                1.0.3
rsa                          4.9
ruamel.yaml                  0.17.21
ruamel.yaml.clib             0.2.6
sacremoses                   0.0.53
safetensors                  0.4.2
scikit-learn                 1.4.0
scipy                        1.12.0
seaborn                      0.13.2
semantic-version             2.10.0
sentence-transformers        3.0.1
sentencepiece                0.1.99
sentry-sdk                   1.24.0
setproctitle                 1.3.2
setuptools                   69.5.1
setuptools-rust              1.6.0
sgmllib3k                    1.0.0
shellingham                  1.5.4
six                          1.16.0
smart-open                   6.3.0
smmap                        5.0.0
sniffio                      1.3.0
socksio                      1.0.0
sounddevice                  0.4.6
SoundFile                    0.9.0.post1
soupsieve                    2.4.1
speech-recognition-fork      3.8.1.2021.6.14
SpeechRecognition            3.10.0
SQLAlchemy                   2.0.25
sqlparse                     0.4.4
sse-starlette                1.8.2
stack-data                   0.6.3
starlette                    0.35.1
statsd                       3.3.0
streamlit                    1.30.0
streamlit-chat               0.0.2.2
svglib                       1.5.1
sympy                        1.12
tenacity                     8.2.3
tensorboard                  2.8.0
tensorboard-data-server      0.6.1
tensorboard-plugin-wit       1.8.1
tensorboardX                 2.6
tensorflow                   2.8.0
tensorflow-estimator         2.12.0
tensorflow-io-gcs-filesystem 0.32.0
tensorstore                  0.1.52
termcolor                    2.3.0
text2vec                     1.1.8
textgen                      0.2.5
texttable                    1.6.7
tf-estimator-nightly         2.8.0.dev2021122109
threadpoolctl                3.2.0
thumbor                      7.4.7
thumbor-plugins-gifv         0.1.2
tiktoken                     0.5.2
timm                         0.9.12
tinycss2                     1.2.1
tls-client                   0.2.1
tokenizers                   0.19.1
toml                         0.10.2
tomli                        2.0.1
tomlkit                      0.12.0
toolz                        0.12.0
torch                        2.1.2
torchvision                  0.16.2
tornado                      6.4
tqdm                         4.66.1
traitlets                    5.14.1
transformer-utils            0.1.1
transformers                 4.41.2
trash-cli                    0.23.11.10
triton                       2.1.0
turtle                       0.0.1
typer                        0.9.0
types-requests               2.31.0.20240106
typing_extensions            4.9.0
typing-inspect               0.9.0
tzdata                       2023.4
tzlocal                      4.3.1
uc-micro-py                  1.0.2
ujson                        5.9.0
uritemplate                  4.1.1
urllib3                      2.1.0
uvicorn                      0.26.0
validators                   0.20.0
wandb                        0.15.3
watchdog                     3.0.0
Wave                         0.0.2
wcwidth                      0.2.13
webcolors                    1.11.1
webencodings                 0.5.1
websocket-client             0.59.0
websockets                   11.0.3
Werkzeug                     3.0.1
wheel                        0.38.4
whisper_jax                  0.0.1
wrapt                        1.14.1
xxhash                       3.2.0
yapf                         0.40.2
yarl                         1.9.4
zhconv                       1.4.3
zhipuai                      2.0.1
zipp                         3.17.0
zope.interface               6.2
zstandard                    0.19.0
zRzRzRzRzRzRzR commented 3 months ago

transformers 4.40.2 试试 torch 2.3.0

Barbery commented 3 months ago

@zRzRzRzRzRzRzR 还是和之前一样，乱输出

包安装情况

# pip install transformers==4.40.2 torch==2.3.0
Looking in indexes: https://mirrors.cloud.tencent.com/pypi/simple
Requirement already satisfied: transformers==4.40.2 in /root/miniconda3/lib/python3.10/site-packages (4.40.2)
Requirement already satisfied: torch==2.3.0 in /root/miniconda3/lib/python3.10/site-packages (2.3.0)
Requirement already satisfied: filelock in /root/miniconda3/lib/python3.10/site-packages (from transformers==4.40.2) (3.13.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /root/miniconda3/lib/python3.10/site-packages (from transformers==4.40.2) (0.23.4)
Requirement already satisfied: numpy>=1.17 in /root/miniconda3/lib/python3.10/site-packages (from transformers==4.40.2) (1.26.3)
Requirement already satisfied: packaging>=20.0 in /root/miniconda3/lib/python3.10/site-packages (from transformers==4.40.2) (23.2)
Requirement already satisfied: pyyaml>=5.1 in /root/miniconda3/lib/python3.10/site-packages (from transformers==4.40.2) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in /root/miniconda3/lib/python3.10/site-packages (from transformers==4.40.2) (2023.12.25)
Requirement already satisfied: requests in /root/miniconda3/lib/python3.10/site-packages (from transformers==4.40.2) (2.31.0)
Requirement already satisfied: tokenizers<0.20,>=0.19 in /root/miniconda3/lib/python3.10/site-packages (from transformers==4.40.2) (0.19.1)
Requirement already satisfied: safetensors>=0.4.1 in /root/miniconda3/lib/python3.10/site-packages (from transformers==4.40.2) (0.4.2)
Requirement already satisfied: tqdm>=4.27 in /root/miniconda3/lib/python3.10/site-packages (from transformers==4.40.2) (4.66.1)
Requirement already satisfied: typing-extensions>=4.8.0 in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (4.9.0)
Requirement already satisfied: sympy in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (1.12)
Requirement already satisfied: networkx in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (3.2.1)
Requirement already satisfied: jinja2 in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (3.1.3)
Requirement already satisfied: fsspec in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (2023.12.2)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (12.1.105)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (12.1.105)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (12.1.105)
Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (8.9.2.26)
Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (12.1.3.1)
Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (11.0.2.54)
Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (10.3.2.106)
Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (11.4.5.107)
Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (12.1.0.106)
Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (2.20.5)
Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (12.1.105)
Requirement already satisfied: triton==2.3.0 in /root/miniconda3/lib/python3.10/site-packages (from torch==2.3.0) (2.3.0)
Requirement already satisfied: nvidia-nvjitlink-cu12 in /root/miniconda3/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch==2.3.0) (12.4.127)
Requirement already satisfied: MarkupSafe>=2.0 in /root/miniconda3/lib/python3.10/site-packages (from jinja2->torch==2.3.0) (2.1.4)
Requirement already satisfied: charset-normalizer<4,>=2 in /root/miniconda3/lib/python3.10/site-packages (from requests->transformers==4.40.2) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /root/miniconda3/lib/python3.10/site-packages (from requests->transformers==4.40.2) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in /root/miniconda3/lib/python3.10/site-packages (from requests->transformers==4.40.2) (2.1.0)
Requirement already satisfied: certifi>=2017.4.17 in /root/miniconda3/lib/python3.10/site-packages (from requests->transformers==4.40.2) (2023.11.17)
Requirement already satisfied: mpmath>=0.19 in /root/miniconda3/lib/python3.10/site-packages (from sympy->torch==2.3.0) (1.3.0)
zRzRzRzRzRzRzR commented 3 months ago

那确实排查不出问题了，没有遇到这种情况，或许要重新拉一次模型，配置文件等都是最新的

generate_kwargs = { "input_ids": model_inputs, "streamer": streamer, "max_new_tokens": max_length, "do_sample": True, "top_p": top_p, "temperature": temperature, "stopping_criteria": StoppingCriteriaList([stop]), "repetition_penalty": 1.2, "eos_token_id": model.config.eos_token_id, } 这样子呢，参考basic_demo/trans_cli_demo.py

Barbery commented 3 months ago

我把github的仓库clone下来，跑了basic_demo/trans_cli_demo.py这个脚本也一样，还是胡乱输出。

Barbery commented 3 months ago

@zRzRzRzRzRzRzR 结贴，最后重新拉了一遍模型后，正常工作了