search

Portkey-AI / gateway

A Blazing Fast AI Gateway with integrated Guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
https://portkey.ai/features/ai-gateway
MIT License
6.35k stars 456 forks source link

Include usage object in openai streaming response by default #691

Open narengogi opened 1 month ago

narengogi commented 1 month ago

What Happened?

To get the usage object, the user has to pass streaming_options: { include_usage: true} This can be default behaviour

What Should Have Happened?

No response

Relevant Code Snippet

No response

Your Twitter/LinkedIn

No response

narengogi commented 6 days ago

tested this for all chat models of openai