search

openai / openai-dotnet

The official .NET library for the OpenAI API
https://www.nuget.org/packages/OpenAI/AbsoluteLatest
MIT License
734 stars 64 forks source link

BUG: `include_usage` implementation doesn't seem to work. #103

Closed janaka closed 3 days ago

janaka commented 4 days ago

This feature seems to be switched on to true by default in the OpenAI client (correct?). However, the StreamingChatCompletionUpdate.Usage property is always null. This appears to be a bug.

Before CompleteChatStreamingAsync(messages, options) is called when I inspect the options object, Stream=null and StreamOptions.IncludeUsage=true. However, immediately after CompleteChatStreamingAsync(messages, options) is called when I inspect the options object, Stream=true and SteamOptions is null.

Client version: Azure OpenAI Azure client version: 2.0.0 Beta 2 OpenAI Client version: implicit (>= 2.0.0-beta.5) OpenAI API version option: AzureOpenAIClientOptions.ServiceVersion.V2024_06_01 OpenAI API provider : Azure OpenAI Model name: GPT-4o

joseharriaga commented 3 days ago

Thank you for reaching out, @janaka ! And thank you for all the useful debug information you provided!

The Azure OpenAI service does not yet support include_usage (a.k.a. stream_options).

Also, for future reference, this repo corresponds to the OpenAI library. If you're using the Azure.AI.OpenAI library instead and are having trouble with it, you can file issues in the Azure repo here: 🔗 https://github.com/Azure/azure-sdk-for-net. Thank you!