search

microsoft / semantic-kernel

Integrate cutting-edge LLM technology quickly and easily into your apps
https://aka.ms/semantic-kernel
MIT License
21.31k stars 3.13k forks source link

.Net: ADR for AI connector support #4577

Closed markwallace-microsoft closed 7 months ago

markwallace-microsoft commented 8 months ago

The purpose of this task is to create an ADR to describe our AI Connector strategy and how new AI Connectors can be contributed.

Below are a series of user stories that must be addressed by the ADR.

Strategy

As a developer using Semantic Kernel I can use AI Connectors that access LLM's deployed in the Cloud e.g. OpenAI, Azure OpenAI, Hugging Face, ... or deployed locally so that I can configure by application to use the optimum LLM or to run completely offline.

  1. Our strategy should allow us to maximise the reuse of AI Connector implementations e.g. if we can configure the same connector to work with online and offline deployments that is a win.
  2. Developers should be able to deploy and use local models easily.

Contributing an AI Connector

As a contributor to Semantic Kernel I can read the documentation which describes how to contribute an AI connector so that I can understand the development process and requirements to have a new AI Connector included in a Semantic Kernel release

  1. Describe how we successfully co-develop AI connectors with the community.
  2. We want to avoid receiving huge PR's that are very difficult to review.
  3. We want to ensure each AI connector includes a defined level of unit tests and integration tests.

P0 LLMs

  1. Phi-2 (Microsoft)
  2. Gemini (Google)
  3. Llama + function calling (Facebook)
  4. Claude (Anthropic)
  5. Multi-modal models (e.g., Stable Diffusion)

Deployment types

  1. Azure AI
  2. Hugging face
  3. Local
  4. Bedrock (Amazon)
Krzysztof318 commented 8 months ago

So... If I understand correctly you prefer connector project per one model (or model family) (even if has multiple deployment types) than project per deployment type where one project contains multiple models?

RogerBarreto commented 8 months ago

As I understand is a Connector per platform/API definition, so a connector can support the same model on a specific platform. i.e:

OpenAI/LMStudio -> Phi-2
HuggingFace -> Phi-2
RogerBarreto commented 8 months ago

@Krzysztof318, will review your PR today and give you a better feedback, is a big one, this one may be graduated to a feature branch later on.

Krzysztof318 commented 8 months ago

@RogerBarreto I would appreciate for dedicated branch, it will also be more convenient for me to maintain small PRs.