Closed nmnduy closed 2 weeks ago
add config option 'max_output_tokens' so we can put 128k tokens in the input while setting the output token to some other value.
for example: gpt-4o input token limit is 128k, while output token limit is 4096
add config option 'max_output_tokens' so we can put 128k tokens in the input while setting the output token to some other value.
for example: gpt-4o input token limit is 128k, while output token limit is 4096