Azure / azure-sdk-for-net

This repository is for active development of the Azure SDK for .NET. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/dotnet/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-net.
MIT License
5.47k stars 4.8k forks source link

OpenAIClient.GetChatCompletionsAsync throws exception #39435

Closed securigy closed 1 year ago

securigy commented 1 year ago

Library name and version

Azure.Search.Document 11.5.0-beta

Describe the bug

I looked at the sample that was built for Azure OpenAI client library for .NET - version 1.0.0-beta.8: https://learn.microsoft.com/en-us/dotnet/api/overview/azure/ai.openai-readme?view=azure-dotnet-preview Specifically the paragraph Use your own data with Azure OpenAI Then, I used it to customize my own function for querying the MS vector database that I already prepared. The code is as follows. It throws exception at the line Response<ChatCompletions>? response = await mOpenAIClient.GetChatCompletionsAsync("gpt-3.5-turbo-0613", chatCompletionsOptions); Function:

    `public async void CreateAzCompletionQuery(string prompt) //, List<string> contextList, CancellationToken? cancellationToken = null)
    {
        using(Log.VerboseCall())
        {
            if (mConfig == null)
                throw new ArgumentException("mConfig is null");
            if (mOpenAIClient == null)
                throw new ArgumentException("mOpenAIClient is null");

            //WebService webService = mConfig.WebServices.Find(s => String.Compare(s.ServiceName, ))
            string selModel = mConfig.Models.SelectedModel;
            if (selModel == null)
                throw new ArgumentException("selModel is null");

            AI.Config.Model? model = mConfig.Models.ModelList.Find(s => String.Compare(s.ModelName, selModel, true) == 0);
            if (model == null)
                throw new ArgumentException(String.Format("model {0} not found", selModel));

            string sysMsg = mConfig.DefaultRole;
            ChatCompletionsOptions chatCompletionsOptions = new ChatCompletionsOptions()
            {
                Messages =
                {
                    new ChatMessage(ChatRole.System, mConfig.DefaultRole),
                    new ChatMessage(ChatRole.User, prompt)
                },

                // The addition of AzureChatExtensionsOptions enables the use of Azure OpenAI capabilities that add to
                // the behavior of Chat Completions, here the "using your own data" feature to supplement the context
                // with information from an Azure Cognitive Search resource with documents that have been indexed.
                AzureExtensionsOptions = new AzureChatExtensionsOptions()
                {
                    Extensions =
                    {
                        new AzureCognitiveSearchChatExtensionConfiguration()
                        {
                            SearchEndpoint = new Uri(azCogSearchEndpoint),
                            IndexName = mIndexName,
                            SearchKey = new AzureKeyCredential(azCogSearchAdminKey),
                        }
                    }
                }
            };

            chatCompletionsOptions.MaxTokens = (model != null) ? model.MaxTokens : mConfig.ParamSet.MaxTokens;
            chatCompletionsOptions.Temperature = (float)mConfig.ParamSet.Temperature;
            chatCompletionsOptions.FrequencyPenalty = (float)mConfig.ParamSet.FrequencyPenalty;
            chatCompletionsOptions.PresencePenalty = (float)mConfig.ParamSet.PresencePenalty;
            chatCompletionsOptions.MaxTokens = mConfig.ParamSet.MaxTokens;
            chatCompletionsOptions.ChoiceCount = mConfig.ParamSet.NumberOfResponses;
            chatCompletionsOptions.NucleusSamplingFactor = (float)mConfig.ParamSet.Topp;

            try
            {

                **Response<ChatCompletions>? response = await mOpenAIClient.GetChatCompletionsAsync("gpt-3.5-turbo-0613", chatCompletionsOptions);**
                if (response != null && !response.GetRawResponse().IsError == false)
                {
                    // Successful result
                    if (response.Value != null && response.Value.Choices != null && response.Value.Choices.Count > 0)
                    {
                        ChatMessage chatMsg = response.Value.Choices[0].Message;
                        Log.VerboseFormat("Role: ${chatMsg.Role}: {chatMsg.Content}");

                        foreach (ChatMessage contextMessage in chatMsg.AzureExtensionsContext.Messages)
                        {
                            // Note: citations and other extension payloads from the "tool" role are often encoded JSON documents
                            // and need to be parsed as such; that step is omitted here for brevity.
                            Console.WriteLine($"{contextMessage.Role}: {contextMessage.Content}");
                        }
                    }
                }
            }
            catch(Exception ex)
            {
                Log.Verbose(ex);
            }
        }
    }

`

The exception is: It basically says that it does not work with this model. So, I tried gpt-3.5-turbo, and gpt-4, but it did not work either....

Azure.RequestFailedException: 'The chatCompletion operation does not work with the specified model, . Please choose different model and try again. You can learn more about which models can be used with each operation here: https://go.microsoft.com/fwlink/?linkid=2197993. Status: 400 (Bad Request) ErrorCode: OperationNotSupported

So what do I wrong here? Why does it throw exception like that?

Expected behavior

No exception

Actual behavior

Throws exception

Reproduction Steps

Run the above code that is basically from the sample...

Environment

WIn11, .NET 4.7, VS2022

securigy commented 1 year ago

Resolved: the name of the model is supposed to be "gpt-35-turbo" instead of "gpt-3.5-turbo"