[Feature Request]: Support Azure OpenAI Service chat completion extensions

[garrytrinder]

Describe your user scenario

A key customer scenario will be to use AI to chat with their own data. Azure OpenAI Service supports bringing your own data from Azure Cognitive Search. The Azure OpenAI Service REST API has an endpoint for chat completion extensions. This extends the chat completion endpoint to enable developers to specify an Azure Cognitive Search data source as part of the request.

Describe the solution you'd like

Extend the config.json schema to include a new type called chat_extensions.

{
    "schema": 1,
    "description": "Chat with your SharePoint data",
    "type": "chat_extensions",
    "completion": {
        "max_tokens": 800,
        "temperature": 0.0,
        "top_p": 1.0,
        "presence_penalty": 0.0,
        "frequency_penalty": 0.0
    }
}

Extend AzureOpenAIPlanner with a new property for setting data sources.

const planner = new AzureOpenAIPlanner({
    apiKey: config.openAIKey,
    apiVersion: '2023-03-15-preview',
    dataSources: [
        {
            type: "AzureCognitiveSearch",
            parameters: {
                endpoint: config.azureCognitiveSearchEndpoint,
                key: config.azureCognitiveSearchKey,
                indexName: config.azureCognitiveSearchIndexName,
            }
        }
    ],
    defaultModel: 'gpt-msteams-spo-oai',
    endpoint: config.openAIEndpoint,
    logRequests: true,
    useSystemMessage: true
});

Requests should be made to the correct endpoint by the SDK.

POST https://{your-resource-name}/openai/deployments/{deployment-id}/extensions/chat/completions?api-version={api-version}

The datasources object should be passed in the body of the request along with the messages object.

{
    "dataSources": [
        {
            "type": "AzureCognitiveSearch",
            "parameters": {
                "endpoint": "https://{acs-resource-name}.search.windows.net",
                "key": "{acs-api-key}",
                "indexName": "{acs-index}"
            }
        }
    ],
    "messages": [
        {
            "role": "user",
            "content": "What are the differences between Azure Machine Learning and Azure AI services?"
        }
    ]
}

Describe alternatives you've considered

The alternative I have considered is to not use the AI capabilities of the SDK, but instead send my own requests and manage the conversation history myself. Here is an example of how I have done this in another sample: https://github.com/garrytrinder/msteams-azure-search-openai-demo

Additional context

Azure OpenAI Service REST API reference - Completions Extensions

[kevmcdonk]

Disappointed to see this on the Backburner. Having your own data is a key feature for some of our clients and frustrating to see this held back

[llowevad]

I agree, would really like to see the teams-ai library incorporate more Azure Open AI especially with using your own data.

I think the library is very helpful. Allows for developers to focus on their intended development purpose instead of having to manage the conversation and payloads for HTTP POSTS or have to incorporate python into their projects. Getting the library to include their own data would get it into enterprise environments much faster.

[Stevenic]

There's another angle to this with supporting functions as well... I feel like both dataSources and functions should be expressed at the prompt level versus the planner. You can only have one planner currently so you need to be able to dynamically swap between prompts that use different data sources and functions. I'll try to have a proposal for how to layer all this stuff in by next week. We're wanting to avoid having to modify sk's config schema as well... So this will likely be in some sort of teams.json file or something... We're also trying to research how SK determines which API (Chat or Completion) to use...

[pkbullock]

I would love this feature, this is only going to get more popular to bring your own data as it dramatically improves the quality and relevance of responses. To be able to surface this within apps would be super powerful. I am working on a project that really could have used this feature.

@garrytrinder I noticed you don't have a license set (or MIT) on your repo example, can I use the sample code? I think I am going to need to switch to raw calls like in that example, Id love to build on what yo have done already.

[garrytrinder]

@pkbullock absolutely, feel free to use it 👍

[martinlaukkanen]

I had a need for this also a little while back and did some quick coding to get it working for a PoC I needed at the time. Now I've come back to this project again and need to get it working for production so I've been reviewing my change in light of the above change #442.

Here's my commit: https://github.com/microsoft/teams-ai/compare/main...martinlaukkanen:teams-ai:main

Basically I just extended the the AzureOpenAIClient.ts with a "dataSources" option (not to be confused with existing DataSource class) which contains the props for AzureAI Search parameters; e.g.

export interface ChatCompletionRequestDataSource {
    type: 'AzureCognitiveSearch' | 'AzureCosmosDB';
    parameters: ChatCompletionRequestDataSourceParameters;
}

and then pass that object to CreateChatCompletionRequest and specifically change the endpoint to call the extensions endpoint;

/openai/deployments/${model}/extensions/chat/...

It works nicely for me, but I can't see how this could be refactored to be used like the DataSources in #442, maybe I'm not seeing how it should be used, but that DataSource is injected to the prompt before the api call, so it provides no way to call the /extensions/ endpoint instead.

Any suggestions on where I should look to extend the existing DataSources for this functionality, or should I go forward with my changes above?