Bug: completion_tokens sometimes exceeds specified max_tokens

stuartleeks / aoai-simulated-api

An exploration into creating a simulated API implementation for Azure OpenAI (AOAI)

MIT License

11 stars 4 forks source link

Closed stuartleeks closed 2 months ago

stuartleeks commented 2 months ago

When calling completions endpoints, the number of generated tokens sometimes exceeds the specified max_tokens value