Bug: GetStreamingChatMessageContentsAsync conflicts with Azure OpenAi studio behind APIM

[gaelix98]

Describe the bug The GetStreamingChatMessageContentsAsync method in Semantic Kernel does not work with Azure OpenAI when the Azure OpenAI endpoint is behind Azure API Management (APIM). If the kernel tries to activate a plugin, a 400 Bad Request error occurs, preventing the interaction from completing. If no plugin is activated, the program runs successfully.

To Reproduce Copy the provided code and run the program. If you use an Azure OpenAI service endpoint behind APIM, you will get the following error:

Unhandled exception. System.ClientModel.ClientResultException: Service request failed. Status: 400 (Bad Request)

at Azure.AI.OpenAI.ClientPipelineExtensions.ProcessMessageAsync(ClientPipeline pipeline, PipelineMessage message, RequestOptions options) at Azure.AI.OpenAI.Chat.AzureChatClient.CompleteChatAsync(BinaryContent content, RequestOptions options) at OpenAI.Chat.ChatClient.<>c__DisplayClass9_0.<g__getResultAsync|0>d.MoveNext() --- End of stack trace from previous location --- at OpenAI.Chat.AsyncStreamingChatCompletionUpdateCollection.AsyncStreamingChatUpdateEnumerator.CreateEventEnumeratorAsync() at OpenAI.Chat.AsyncStreamingChatCompletionUpdateCollection.AsyncStreamingChatUpdateEnumerator.System.Collections.Generic.IAsyncEnumerator.MoveNextAsync() 2 at Microsoft.SemanticKernel.Connectors.OpenAI.ClientCore.GetStreamingChatMessageContentsAsync(String targetModel, ChatHistory chatHistory, PromptExecutionSettings executionSettings, Kernel kernel, CancellationToken cancellationToken)+MoveNext()

Expected behavior The program should print a greeting as a response.

Platform

• OS: Windows
• IDE: Visual Studio
• Language: C#
• Source: [ "Microsoft.SemanticKernel" Version="1.21.1" ]

Additional context The same program works correctly with an Azure OpenAI service that is not under APIM. Additionally, it works with version 1.15 of Microsoft Semantic Kernel when running under APIM.

Code Main

private static async Task Main(string[] args)
    {
        Kernel kernel = ConfigureKernel(); //add a simple plugin with a string return
        var chatCompletionService = kernel.GetRequiredService<IChatCompletionService>();

        ChatHistory chatHistory = new ChatHistory();

        // Add system message to chat history
        chatHistory.AddSystemMessage("activate the plugin");

        // Add previous conversation history to chat history
        chatHistory.Add(new ChatMessageContent(AuthorRole.User, "Hello"));

        // Set OpenAI prompt execution settings
        AzureOpenAIPromptExecutionSettings openAIPromptExecutionSettings = new()
        {
            MaxTokens = 3000,
            ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions,
            Temperature = 0.1,
            FrequencyPenalty = 1
        };

        // Retrieve streaming chat messages asynchronously
        var response = chatCompletionService.GetStreamingChatMessageContentsAsync(
            chatHistory,
            executionSettings: openAIPromptExecutionSettings,
            kernel: kernel);

        string combinedResponse = "";
        await foreach (var message in response)
        {
            combinedResponse += message;
        }

        Console.WriteLine(combinedResponse);
    }

public static Kernel ConfigureKernel()
        {
            var AOAI_API_ENDPOINT=
            var AOAI_GPT_MODEL= 
           var AOAI_API_KEY=

            var builder = Kernel.CreateBuilder();
            var clientOptions = new AzureOpenAIClientOptions();
            clientOptions.NetworkTimeout = TimeSpan.FromMinutes(10);
            AzureOpenAIClient azureOpenAIClient = new AzureOpenAIClient(endpoint: new Uri(AOAI_API_ENDPOINT), credential: new AzureKeyCredential(AOAI_API_KEY), options: clientOptions);
            builder.AddAzureOpenAIChatCompletion(azureOpenAIClient:azureOpenAIClient,
                    deploymentName: AOAI_GPT_MODEL
                     );
            builder.AddAzureOpenAIChatCompletion(AOAI_GPT_MODEL,azureOpenAIClient);

            builder.Services.AddSingleton<AzureOpenAIClient>(azureOpenAIClient);
            TestPlugin testPlugin = new TestPlugin();
            builder.Plugins.AddFromObject(testPlugin);
            var kernel = builder.Build();
            return kernel;
        }

Plugin:

public class TestPlugin
{

    public TestPlugin()
    {

    }

    [KernelFunction, Description("use this function to greet the user")]
    public async Task<string> ChitChat(Kernel kernel)
    {

        return "hi";
    }
}

[RogerBarret0]

By any chance, when you use the AzureOpenAIClient directly (Azure.AI.OpenAI SDK), do you get the same problem? If you haven't tried, I would ask to do so, this potentially might be a bug from their SDK, but I need to validate this information before opening a ticket on Azure AI SDK repo with the reproduction code.

[gaelix98]

We have no problem when using semantic kernel with azureopenai client under apim when no plugin is invoked. Everything works fine and i can chit chat, but as soon as i activate a plugin, i get the error above