What does the parameter "max_seq_len" do exactly?

luogen1996 / LaVIN

[NeurIPS 2023] Official implementations of "Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Models"

509 stars 38 forks source link

What does the parameter "max_seq_len" do exactly? #8

Closed WizardMx closed 1 year ago

WizardMx commented 1 year ago

It seems this para infects inference time very much

luogen1996 commented 1 year ago

"max_seq_len" determines the sequence length of the batch inference. Decreasing this value will greatly save the inference time, but may lead to incomplete response.