Add LLM Connector for Azure OpenAI Models

BCG-X-Official / artkit

Automated prompt-based testing and evaluation of Gen AI applications

https://bcg-x-official.github.io/artkit/_generated/home.html

Apache License 2.0

114 stars 17 forks source link

Add LLM Connector for Azure OpenAI Models #64

Open tezansahu opened 1 month ago

tezansahu commented 1 month ago

Justification

The Azure OpenAI API is compatible with OpenAI's API. With Azure OpenAI, one can set up their own deployments of the common GPT and Codex models. When calling the API, they need to specify the deployment they want to use.

Details

The new class will live in src/artkit/model/llm/azureopenai/

Be sure to update the docs:

README: https://github.com/BCG-X-Official/artkit?tab=readme-ov-file#supported-model-providers
Contributing: https://github.com/BCG-X-Official/artkit/blob/1.0.x/sphinx/source/user_guide/introduction_to_artkit/connecting_to_genai_models.ipynb
Connecting to Gen AI Models: https://github.com/BCG-X-Official/artkit/blob/1.0.x/sphinx/source/user_guide/introduction_to_artkit/connecting_to_genai_models.ipynb
.env_example: https://github.com/BCG-X-Official/artkit/blob/1.0.x/.env_example

Please see the tutorial: Creating New Model Classes

seanggani commented 1 month ago

Thank you for raising this issue! We agree this would be a good enhancement

seanggani commented 1 month ago

Hi @tezansahu (or anyone else interested), we would love to invite you to contribute here! For reference, we created a similar connector for Amazon's titan bedrock model in this file and would be happy to provide more guidance on this.

matthew-wong-bcg commented 1 week ago

I've added additional details about the requirements:

Issue

As an extension to ARTKIT, we would like the package to be able to support OpenAI models deployed on Azure.

Solution

Leverage the OpenAIChat object implementation here located here and implement a similar class called _azureopenai.py located in src/artkit/model/llm/azure
Use the existing paradigms within the _openai.py implementation but instead, import the AsyncAzureOpenAI SDK instead of the AsyncOpenAI SDK. The source for AsyncAzureOpenAI is located here
Update the RELEASE_NOTES.rst file:
- Increment a minor version here

Notes

There are differences between the inference parameters between OpenAI and Azure OpenAI - some of those differences are located in the Azure SDK documentation located here