Closed MakotoE closed 2 months ago
CreateChatCompletionRequest.max_tokens is a Option<u16> as of 0.23.1.
CreateChatCompletionRequest.max_tokens
Option<u16>
The newer models such as gpt-4o has a context window of 128,000 tokens. This context window limit is the sum of input and output tokens.
I believe the max_tokens field should be Option<u32> to allow numbers as high as 128,000.
max_tokens
Option<u32>
Thank you for reporting the bug, a PR is most welcome!
CreateChatCompletionRequest.max_tokens
is aOption<u16>
as of 0.23.1.The newer models such as gpt-4o has a context window of 128,000 tokens. This context window limit is the sum of input and output tokens.
I believe the
max_tokens
field should beOption<u32>
to allow numbers as high as 128,000.