For historical reasons, we set max_tokens to 1024 for all LLMs by default. This would often cause unexpected failures when we benchmark datasets which require more than 1K tokens as responses. With this CL, we opt in the default behavior defined by each LLM, usually meaning the longest tokens possible for generation.
Side effect:
All previous benchmarks will have a different hash due to the default LLM sampling options change.
Langfun LLMs to set
max_tokensto None by defaultFor historical reasons, we set
max_tokensto 1024 for all LLMs by default. This would often cause unexpected failures when we benchmark datasets which require more than 1K tokens as responses. With this CL, we opt in the default behavior defined by each LLM, usually meaning the longest tokens possible for generation.Side effect: