Add support for Azure OpenAI features

[elwc-edge]

Hi, I'm a user of ChatBot-UI 2 project and I think it's awesome. I'm wondering if you could add support for some of the Azure OpenAI features, such as:

• DALL-E: A model that can generate text and images from natural language prompts. This would enable more creative and expressive chatbots that can respond with both words and pictures. For more information, see "DALL-E 3: Text and Image Generation with Azure OpenAI)".

• Enhance Configuration: A feature that allows you to configure multiple deployments and alternative model names for your chatbot. This would enable more flexibility and customization for different use cases and scenarios. For more information, see "Enhance Configuration to Allow Multiple Deployments & Alternative Model Names".

• Embeddings: A feature that allows you to leverage semantic embeddings to improve the quality and relevance of your chatbot responses. This would enable more natural and coherent chatbots that can understand the context and intent of the user. For more information, see "Azure OpenAI Embeddings".

• GPT-4-Vision/GPT-V: A model that can generate text and images from natural language prompts, and also answer questions about the images. This would enable more advanced and interactive chatbots that can create and understand multimodal content. For more information, see “How to use the GPT-4 Turbo with Vision model” and “Quickstart: Use GPT-4 Turbo with Vision on your images and videos with the Azure Open AI Service”.

References: DALL-E 3: Text and Image Generation with Azure OpenAI Azure OpenAI Embeddings GPT-4 and GPT-4 Turbo Preview Models How to use the GPT-4 Turbo with Vision model Quickstart: Use GPT-4 Turbo with Vision on your images and videos with the Azure Open AI Service

[mckaywrigley]

We want to support more of those things for sure, but for now we do support text inference if you open your profile settings and enter the Azure config settings.

[Chr96er]

@mckaywrigley I was able to make requests to Dall-E if a message starts with "Generate an image" by making some adjustments in app/api/chat/openai/route.ts. I send the prompt to openai chat completions first to make it a lot more elaborate. I'm getting the response (either URL or base64) but I'm unable to get the image to render in the chat. I can return the image URL using the code below which isn't too bad either but it would be great to display the image in the UI.

return new Response(response.data[0].url, {
  status: 200
})

Would you be able to point me in the right direction for making either the URL or base64 render? Do I first need to somehow use uploadMessageImage?

edit: You can find my code in my fork.