search

Azure / azure-sdk-for-net

This repository is for active development of the Azure SDK for .NET. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/dotnet/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-net.
MIT License
5.46k stars 4.8k forks source link

[BUG] Token limitations don't seem to get thrown from the client library #46782

Open adameska opened 1 week ago

adameska commented 1 week ago

Library name and version

Azure.AI.OpenAI

Describe the bug

When trying to call a chat completion it seems like if there is an error nothing is ever thrown and it appears like the application just sits. I have a controller action that calls with a user request that catches a generic exception and the catch is never hit nor is anything ever returned. I was hitting the token limit (the only way i knew this was replicating the messages in the playground) and had no idea. Here is a very small snippet of the code illustrating the line causing it.

ChatClient chatClient = _azureOpenAIClient.GetChatClient("gpt-4o-mini"); chatClient.CompleteChat(conversationMessages, options); //this never returns, never throws, i have no idea if this is a slow request or it immediately crashed. conversationMessages is over the token limit.

Expected behavior

An meaningful exception should be thrown if an error occurred.

Actual behavior

Call never returns anything.

Reproduction Steps

ChatClient chatClient = _azureOpenAIClient.GetChatClient("gpt-4o-mini"); //some large request in conversation messages chatClient.CompleteChat(conversationMessages, options); //this never returns, never throws, i have no idea if this is a slow request or it immediately crashed. conversationMessages is over the token limit.

Environment

Windows 11 .netcore 8 VS 17.11

github-actions[bot] commented 1 week ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jpalvarezl @ralph-msft @trrwilson.