search

momegas / megabots

🤖 State-of-the-art, production ready LLM apps made mega-easy, so you don't have to build them from scratch 🤯 Create a bot, now 🫵
MIT License
341 stars 35 forks source link

support azure openai api #4

Open timlincool opened 1 year ago

timlincool commented 1 year ago

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

I have an azure openai api account, but I don't see where it can be used

Describe the solution you'd like A clear and concise description of what you want to happen.

If not, can you access the azure openai api?

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

It may be possible to add a proxy, but I haven't tested it https://github.com/stulzq/azure-openai-proxy

Additional context Add any other context or screenshots about the feature request here.

momegas commented 1 year ago

Haven't searched through this specific API. Do I need any special access? Can you point me to some relevant documentation?

timlincool commented 1 year ago

Thank you for your hard work on this project! I was wondering if it's possible to add support for Azure OpenAI API. I noticed that Auto-GPT supports Azure API, which might be a helpful reference for implementation. Here's the GitHub repository link for Auto-GPT: https://github.com/Significant-Gravitas/Auto-GPT

I hope this feature can be considered, and I'm looking forward to seeing the project grow. Thank you!

momegas commented 1 year ago

Thank you for your kind words. I'll have a look 😁 A post in any community you take part in would help a lot the project grow.

Blubbaa commented 1 year ago

I think this feature would be very helpful and could also greatly extend the number of potential users of this library.

I have already tested to switch the endpoints to Azure OpenAI, but it involved some "hacking" in the library code, beside setting the respective environment variables. If needed I can share exactly what I have changed to make it work, but in general I don't like the approach since it's quite hacky and the integration should be done more thoughtfully I think. As a summary it involved setting these environment variables:

os.environ["OPENAI_API_TYPE"] = "azure"
os.environ["OPENAI_API_KEY"] = "661xxxxxxxxxxxxxxxxxxxxx80c8"
os.environ["OPENAI_API_BASE"] = "https://xxxxxxxxx.openai.azure.com/"
os.environ["OPENAI_API_VERSION"] = "2023-05-15"

And then adding the AzureChatOpenAI LLM from langchain in the select_model function, as well as setting the deployment arg for the OpenAIEmbeddings models.

@momegas, here is some more information on differences between OpenAI and Azure OpenAI enpoints: https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/switching-endpoints