How to set max_new_tokens

[treya-lin]

Hello thanks for your great work.

I am trying finetuning glm3 with lora, but I am a bit confused by the difference between max_output_length and the max_new_tokens. I read the finetuning code and noticed only max_output_length and max_input_length are used when cutting input data . So what does max_new_tokens do ? Do I need to make sure it is larger than the longest turn of conversation in my data?

say the data have many turns and the token number of each turn is like this:

user: ... (len:53)
asssistant: ...(len:123)
user: ...(len:150)
assistant: ...(len:100) 

so Does it mean, before I finetune the model, I need to make sure the max_new_tokens should be set greater than the longest assistant turn in my data(i.e. 123)?

Is my understanding correct? Any clarification is greatly appreciated!!

[zRzRzRzRzRzRzR]

max_output_length 按照transformers传递参数的方式就行了，都是通用的

[treya-lin]

max_output_length 按照transformers传递参数的方式就行了，都是通用的

我是想问 generation_config 里面的 max_new_token ，不是 max_output_length (https://github.com/THUDM/ChatGLM3/blob/main/finetune_demo/configs/lora.yaml)