search

Azure / azure-sdk-for-net

This repository is for active development of the Azure SDK for .NET. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/dotnet/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-net.
MIT License
5.56k stars 4.82k forks source link

[FEATURE REQ] Azure.AI.OpenAI `include_usage` in chat completion options #44237

Open janaka opened 6 months ago

janaka commented 6 months ago

Library name

Azure.AI.OpenAI

Please describe the feature.

Add include_usage options for streaming chat completions. Relevant OpenAI docs below.

https://platform.openai.com/docs/api-reference/completions/create

include_usage

Request:

image

Respose:

image
github-actions[bot] commented 6 months ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jpalvarezl @trrwilson.

BlackGad commented 6 months ago

https://github.com/Azure/azure-rest-api-specs/issues/25062 related stuff

janaka commented 5 months ago

Updated to the new v2 beta2 client. This feature seems to be switched to true by default in the OpenAI client but doesn't seem to be working.

https://github.com/openai/openai-dotnet/issues/103

Mako217 commented 3 months ago

Are there any updates on this?

janaka commented 3 months ago

Are there any updates on this?

@Mako217 on https://github.com/openai/openai-dotnet/issues/103 I was told the backend doesn't have the feature yet. Generally Azure have been terrible at communicating what the state of these features are. There are at least two other features we are waiting for were the backend state is unclear.

kenneth-stylespace commented 2 months ago

Seems it is now supported. Tested working on chat completion with OpenAI npm SDK with Azure credential, with ChatGPT-4o-0806, API version 2024-08-01-preview.

It never works in previous versions.

janaka commented 2 months ago

yes, confirmed for the save model and rest API version against eastus and sweden deployments. streaming + include_usage works for me with Postman.

janaka commented 1 month ago

We are not on v2.1 of the dotnet SDK and this issue still has not been fixed.

As documented in https://github.com/openai/openai-dotnet/issues/103 this is a bug in the client. The backend is confirmed to support this feature.

Can somebody on the SDK team at least acknowledge this is an issue cc @jpalvarezl @trrwilson .