search

THUDM / ChatGLM3

ChatGLM3 series: Open Bilingual Chat LLMs | 开源双语对话语言模型
Apache License 2.0
13.5k stars 1.57k forks source link

TypeError: ChatGLMTokenizer._pad() got an unexpected keyword argument 'padding_side' #1324

Open jeejeelee opened 1 month ago

jeejeelee commented 1 month ago

System Info / 系統信息

torch==2.4.0 transformers==4.45.0

Who can help? / 谁可以帮助到您？

No response

Information / 问题信息

Reproduction / 复现过程

Step1

升级到transformers到4.45.0

Step2

运行如下代码：


from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm3-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("THUDM/chatglm3-6b", trust_remote_code=True).half().cuda()
model = model.eval()
response, history = model.chat(tokenizer, "你好", history=[])
print(response)
response, history = model.chat(tokenizer, "晚上睡不着应该怎么办", history=history)
print(response)

Expected behavior / 期待表现

应该需要更新HF仓库中的:ChatGLMTokenizer._pad 的参数

jeejeelee commented 1 month ago

@zRzRzRzRzRzRzR 辛苦花点时间看下这个问题，glm4也存在类似问题，非常感谢~

lilqz66 commented 1 month ago

import torch from datasets import Dataset from diffusers import FluxPipeline, KolorsPipeline from PIL import Image # 导入PIL库用于保存图片

from yijian_community.defense import InternVL2ImageDefense, ThuCoaiShieldLM from yijian_community.model import HFTxt2ImgInfer

def txt2img_zh_usage_example(txt2img_zh_model="/mnt/public/lqz/prompt_attack/Kolors-diffusers"): txt2img_zh = HFTxt2ImgInfer( model_path=txt2img_zh_model, pipe=KolorsPipeline, variant="fp16", )

# 生成一张图片
text_prompt = "今天天气很好。"
img = txt2img_zh.infer_data(
    data=text_prompt, guidance_scale=5.0, num_inference_steps=50
)

# 保存图片到指定路径
output_path = "test.jpg"
img.save(output_path)  # 保存图片

print(f"图片已保存到 {output_path}")

txt2img_zh_usage_example()

kolor文本好像用的是chaglm3，现在也会报错： TypeError: ChatGLMTokenizer._pad() got an unexpected keyword argument 'padding_side'

环境： Package Version

accelerate 0.34.2 aiohappyeyeballs 2.4.0 aiohttp 3.10.5 aioprometheus 23.12.0 aiosignal 1.3.1 annotated-types 0.7.0 anyio 4.5.0 async-timeout 4.0.3 attrs 24.2.0 bert-score 0.3.13 certifi 2024.8.30 charset-normalizer 3.3.2 click 8.1.7 cloudpickle 3.0.0 colorama 0.4.6 contourpy 1.3.0 cycler 0.12.1 datasets 3.0.0 diffusers 0.30.3 dill 0.3.8 diskcache 5.6.3 distro 1.9.0 einops 0.8.0 exceptiongroup 1.2.2 fastapi 0.115.0 filelock 3.16.1 fonttools 4.53.1 frozenlist 1.4.1 fsspec 2024.6.1 gguf 0.10.0 h11 0.14.0 httpcore 1.0.5 httptools 0.6.1 httpx 0.27.2 huggingface-hub 0.25.0 idna 3.10 importlib_metadata 8.5.0 iniconfig 2.0.0 interegular 0.3.3 jieba 0.42.1 Jinja2 3.1.4 jiter 0.5.0 joblib 1.4.2 jsonschema 4.23.0 jsonschema-specifications 2023.12.1 kiwisolver 1.4.7 lark 1.2.2 llvmlite 0.43.0 lm-format-enforcer 0.10.6 lxml 5.3.0 markdown-it-py 3.0.0 MarkupSafe 2.1.5 matplotlib 3.9.2 mdurl 0.1.2 mistral_common 1.4.3 mpmath 1.3.0 msgpack 1.1.0 msgspec 0.18.6 multidict 6.1.0 multiprocess 0.70.16 nest-asyncio 1.6.0 networkx 3.3 ninja 1.11.1.1 numba 0.60.0 numpy 1.26.4 nvidia-cublas-cu12 12.1.3.1 nvidia-cuda-cupti-cu12 12.1.105 nvidia-cuda-nvrtc-cu12 12.1.105 nvidia-cuda-runtime-cu12 12.1.105 nvidia-cudnn-cu12 9.1.0.70 nvidia-cufft-cu12 11.0.2.54 nvidia-curand-cu12 10.3.2.106 nvidia-cusolver-cu12 11.4.5.107 nvidia-cusparse-cu12 12.1.0.106 nvidia-ml-py 12.560.30 nvidia-nccl-cu12 2.20.5 nvidia-nvjitlink-cu12 12.6.68 nvidia-nvtx-cu12 12.1.105 openai 1.50.0 orjson 3.10.7 outlines 0.0.46 packaging 24.1 pandas 2.2.2 partial-json-parser 0.2.1.1.post4 pillow 10.4.0 pip 24.2 pluggy 1.5.0 portalocker 2.10.1 prometheus_client 0.21.0 prometheus-fastapi-instrumentator 7.0.0 protobuf 5.28.2 psutil 6.0.0 py-cpuinfo 9.0.0 pyairports 2.1.1 pyarrow 17.0.0 pycountry 24.6.1 pydantic 2.9.2 pydantic_core 2.23.4 Pygments 2.18.0 pyparsing 3.1.4 pytest 8.3.3 python-dateutil 2.9.0.post0 python-dotenv 1.0.1 pytz 2024.2 PyYAML 6.0.2 pyzmq 26.2.0 quantile-python 1.1 ray 2.36.0 referencing 0.35.1 regex 2024.9.11 requests 2.32.3 rich 13.8.1 rouge-metric 1.0.1 rpds-py 0.20.0 sacrebleu 2.4.3 safetensors 0.4.5 scikit-learn 1.5.2 scipy 1.14.1 sentence-transformers 3.1.1 sentencepiece 0.2.0 setuptools 59.6.0 six 1.16.0 sniffio 1.3.1 starlette 0.38.5 sympy 1.13.3 tabulate 0.9.0 threadpoolctl 3.5.0 tiktoken 0.7.0 timm 1.0.9 tokenizers 0.20.0 tomli 2.0.1 torch 2.4.0 torchvision 0.19.0 tqdm 4.66.5 transformers 4.45.1 triton 3.0.0 typing_extensions 4.12.2 tzdata 2024.1 urllib3 2.2.3 uvicorn 0.30.6 uvloop 0.20.0 vllm 0.6.2 watchfiles 0.24.0 websockets 13.0.1 xformers 0.0.27.post2 xxhash 3.5.0 yarl 1.11.1 yijian-community 0.1.4 zipp 3.20.2

LeiyanGithub commented 1 month ago

chatglm3-6b-32k报错

image
bjr-yql commented 1 month ago

kolors也使用了chatglm也存在这个问题，把transformers库的版本下降到Version: 4.34.0可以解决这个问题

LeiyanGithub commented 1 month ago

kolors也使用了chatglm也存在这个问题，把transformers库的版本下降到Version: 4.34.0可以解决这个问题 请问只是降低了版本就解决了么？我这边尝试发现还是没法解决呢

jeejeelee commented 1 month ago

kolors也使用了chatglm也存在这个问题，把transformers库的版本下降到Version: 4.34.0可以解决这个问题 请问只是降低了版本就解决了么？我这边尝试发现还是没法解决呢

4.43.0 应该可以解决这个问题。 PS： 建议遇到类似问题的，可以去GLM4的repo提issue，可能会响应更快点(GLM4也存在类似的问题)

huolongguo1O commented 1 month ago

modelscope上说了是chatglm不支持最新版本

LeiyanGithub commented 1 month ago

kolors也使用了chatglm也存在这个问题，把transformers库的版本下降到Version: 4.34.0可以解决这个问题 请问只是降低了版本就解决了么？我这边尝试发现还是没法解决呢

4.43.0 应该可以解决这个问题。 PS： 建议遇到类似问题的，可以去GLM4的repo提issue，可能会响应更快点(GLM4也存在类似的问题)

之前的问题解决了，貌似出现了新的问题，vllm不支持这个版本！ image

zRzRzRzRzRzRzR commented 1 month ago

已经收到这个问题，预计节后投入人力解决

countzero commented 1 month ago

Hi,

I ran into the same error in the context of llama.cpp trying to convert https://huggingface.co/THUDM/glm-4-9b-chat to GGUF.

Error

TypeError: ChatGLM4Tokenizer._pad() got an unexpected keyword argument 'padding_side'

Workaround

Downgrade the transformers package to version 4.44.2:

pip uninstall transformers -y; pip install transformers==4.44.2

It seems as if this bug is introduced with the transformers release v4.45.0.

I hope this helps someone :wink:

wuhanzhuyang commented 1 month ago

降低版本到4.40.2，可以了 pip uninstall transformers -y; pip install transformers==4.40.2

Orenji-Tangerine commented 1 month ago

Can Transformers 4.45.0 run without errors now? If not, any plan to address this?

BLANKBLK commented 1 month ago

换成4.40会出现新的错误，怎么办？ ModuleNotFoundError: No module named 'transformers.models.phi3'

huolongguo1O commented 1 month ago

换成4.40会出现新的错误，怎么办？ ModuleNotFoundError: No module named 'transformers.models.phi3'

换4.42 或移除phi3相关代码

AGXAI commented 1 month ago

Hi guys I have the same issue recently, even though i mainly use FLUX, I recently switched to KOLORS, which is literally MJV6.1 killer with lower CFG and samplers, But I have this issue of the GLM thing, I tried everything and its not working! pls help!

AGXAI commented 1 month ago

We have received this question and plan to invest manpower to solve it after the holiday.

please!!!!!

huolongguo1O commented 1 month ago

Hi guys I have the same issue recently, even though i mainly use FLUX, I recently switched to KOLORS, which is literally MJV6.1 killer with lower CFG and samplers, But I have this issue of the GLM thing, I tried everything and its not working! pls help!

Same as this issue?

AGXAI commented 1 month ago

Hi,

I ran into the same error in the context of llama.cpp trying to convert https://huggingface.co/THUDM/glm-4-9b-chat to GGUF.

Error

TypeError: ChatGLM4Tokenizer._pad() got an unexpected keyword argument 'padding_side'

Workaround

Downgrade the transformers package to version 4.44.2:

pip uninstall transformers -y; pip install transformers==4.44.2

It seems as if this bug is introduced with the transformers release v4.45.0.

I hope this helps someone 😉

unfortunately it didn't help bro! thank you though

Foluolaaa commented 1 month ago

可以参考这个解决方案 https://huggingface.co/THUDM/LongWriter-glm4-9b/commit/778b5712634889f5123d6c463ca383bc6dd5c621#d2h-069285

ztsweet commented 1 week ago

+1

eaidova commented 1 week ago

I tried to apply this fix https://huggingface.co/THUDM/chatglm3-6b/discussions/64 it works on my side