.Net: KernelPlugin is not working AzureOpenAIChatCompletionWithDataConfig

[rupeshtech]

KernelPlugin is not working AzureOpenAIChatCompletionWithDataConfig I have an application which uses semantic kernel with plugins.

Requirement Requirement is application gets the data using own data with Azure OpenAI at the same time works with Plugins

Working code following piece of code is working fine with plugins

builder.Services.AddKernel().AddAzureOpenAIChatCompletion(
             "gpt432k",
             "https://xxx.openai.azure.com/",
             "xxxx"
             );
 builder.Services.AddSingleton<KernelPlugin>(sp => KernelPluginFactory.CreateFromType<WeatherPlugin>(serviceProvider: sp));

Not Working code the following piece of code not working with plugins

var config = new AzureOpenAIChatCompletionWithDataConfig
{
    CompletionApiKey = "xxx",
    CompletionApiVersion = "2023-08-01-preview",
    CompletionEndpoint = "https://xxx.openai.azure.com",
    CompletionModelId = "gpt432k",
    DataSourceApiKey = "xxxx",
    DataSourceEndpoint = "https://xxx.search.windows.net",
    DataSourceIndex = "xx-index"
};
builder.Services.AddKernel().AddAzureOpenAIChatCompletion(
             config:config
             );
builder.Services.AddSingleton<KernelPlugin>(sp => KernelPluginFactory.CreateFromType<WeatherPlugin>(serviceProvider: sp));

Controller Code (Api) Following is the code in the controller

 var result = await this._chatCompletionService.GetChatMessageContentAsync
                (
                chatHistory: history,
                executionSettings: openAIPromptExecutionSettings,
                kernel: this._kernel
                );

Summary When I use AzureOpenAIChatCompletionWithDataConfig with AddAzureOpenAIChatCompletion it doesn't work with plugins only get the data with now data source.

[rupeshtech]

@markwallace-microsoft @matthewbolanos any update on the issue?

[stephentoub]

Requirement is application gets the data using own data with Azure OpenAI at the same time works with Plugins

This is not supported by Azure today: https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/openai/Azure.AI.OpenAI/README.md#use-chat-functions "NOTE: The concurrent use of Chat Functions and Azure Chat Extensions on a single request is not yet supported. Supplying both will result in the Chat Functions information being ignored and the operation behaving as if only the Azure Chat Extensions were provided. To address this limitation, consider separating the evaluation of Chat Functions and Azure Chat Extensions across multiple requests in your solution design."

[dmytrostruk]

@rupeshtech Here is a new recommended way how to use chat completion with data functionality: https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/ChatCompletion/AzureOpenAIWithData_ChatCompletion.cs

As mentioned in previous comment, using chat completion with data together with function calling is not supported by Azure .NET SDK today. In case you have further questions, feel free to re-open this issue or create a new one. Thanks a lot!