search

microsoft / autogen

A programming framework for agentic AI 🤖
https://microsoft.github.io/autogen/
Creative Commons Attribution 4.0 International
32.46k stars 4.73k forks source link

Add the ability to use models deployed on Azure AI Studio #3902

Open rohanthacker opened 1 week ago

rohanthacker commented 1 week ago

What feature would you like to be added?

I would like the ability to use a model that is deployed on Azure AI Studio and uses the Azure AI Model Inference API.

If needed, I would like to assist in the creation of the feature. However I have a few questions and require some help about what would be the best way to implement this feature.

Questions:

  1. Since the Azure AI Model Inference API is compatible with Azure OpenAI model deployments can we extend the already created AzureOpenAIChatCompletionClient?

I have already tried to do this however the API produces an invalid URL and responds with a 404 error, as the endpoint created by Azure AI Studio and the client are not the same.

  1. Or would it be preferred to extend the ChatCompletionClient, I started with this initially but I noticed a fair bit of overlap with the OpenAI Clients.
  1. Or perhaps there is a simpler way we can integrate this API that I am not aware of?

Looking forward to discussing more on this

Why is this needed?

Azure AI Studio provides a large catalog of models along with various deployment options that make it easy for developers to access a wide variety of models. Given the nature of this project, having the ability to integrate this diverse set of models out of the box will allow for more adoption of the project and allow developers to bring their own model in without the need to code a new client for each.

jackgerrits commented 1 week ago

Thanks for the issue! I think supporting the Azure AI Model Inference API would be great! I think it makes sense to have it as a separate model client that also implements the ChatCompletionClient protocol makes sense.

We'd love if you're interested in helping build this!

rohanthacker commented 1 week ago

@jackgerrits I'll be happy to implement these changes, can this task be assigned to me as I have already started work on this task in my fork of this repository. I'll raise a draft pull request in a day or so for us to discuss.

rysweet commented 5 days ago

@rohanthacker - this is supported in dotnet now with https://github.com/microsoft/autogen/pull/3790