Enhancement: Send `"stream_options": {"include_usage": true}` when `"stream": true`

danny-avila / LibreChat

Enhanced ChatGPT Clone: Features Anthropic, AWS, OpenAI, Assistants API, Azure, Groq, o1, GPT-4o, Mistral, OpenRouter, Vertex AI, Gemini, Artifacts, AI model switching, message search, langchain, DALL-E-3, ChatGPT Plugins, OpenAI Functions, Secure Multi-User System, Presets, completely open-source for self-hosting. Actively in public development.

https://librechat.ai/

MIT License

19.15k stars 3.19k forks source link

Enhancement: Send `"stream_options": {"include_usage": true}` when `"stream": true` #4099

Open Manouchehri opened 2 months ago

Manouchehri commented 2 months ago

What features would you like to see added?

When "stream": true, it'd be ideal if LibreChat could also send "stream_options": {"include_usage": true}. This only applies to OpenAI API endpoints.

More details

When running through a proxy, it's nice to get the token usage cached too.

Which components are impacted by your request?

General

Pictures

No response

Code of Conduct

[X] I agree to follow this project's Code of Conduct