search

openai / openai-openapi

OpenAPI specification for the OpenAI API
https://platform.openai.com/docs/api-reference/introduction
MIT License
1.23k stars 357 forks source link

Token Limit for gpt-4o-mini Model in v1/chat/completions API #319

Open kerlw opened 2 weeks ago

kerlw commented 2 weeks ago

I have been testing the gpt-4o-mini model using the v1/chat/completions API and have encountered an issue regarding the token limit. According to the official documentation, the context window for the gpt-4o-mini model is specified as 128,000 tokens. However, in my tests, the total token length limit seems to be restricted to approximately 16,000 tokens instead.

Here are the details of my test:

Model: gpt-4o-mini Test Scenario: I included a history conversation of around 16,000 tokens and set max_tokens to 4,000. Issue: The response was truncated, and I only received a few hundred tokens of output. model in response is gpt-4o-mini-2024-07-18. The stop_reason: length in the response indicates that the length limit was exceeded, despite the remaining difference between 16,000 tokens and the documented 128,000 tokens being substantial. I would appreciate it if you could investigate this discrepancy and confirm whether there are any additional limitations or if this might be an issue with the API. Your assistance in clarifying this matter would be greatly valued.

Thank you.

kerlw commented 2 weeks ago

Current tier Usage tier 3