THUDM / ChatGLM2-6B

ChatGLM2-6B: An Open Bilingual Chat LLM | 开源双语对话语言模型
Other
15.71k stars 1.85k forks source link

[Help] 请问api.py中的max_length参数具体指的是什么length #627

Open zhengyangyong opened 11 months ago

zhengyangyong commented 11 months ago

Is there an existing issue for this?

Current Behavior

想确定是如下哪一种:

  1. 输入(Token)长度
  2. 输出(Token)长度
  3. 上下文长度
  4. 输入+输出长度

现象如图:

d2deb6f6b6fd1dcfe3c74c0512aab23

这个参数,如果设置得比输入小会报错:Input length of input_ids is 2183, but max_length is set to 500. This can lead to unexpected behavior. You should consider increasing max_new_tokens

从这个表现可以看出这个参数是限制输入长度的,但是很多模型此参数是 限制输出长度 的,如果限制输出长度,又应该如何设置

Expected Behavior

Steps To Reproduce

Environment

Anything else?