[Bug] Unable to switch between models when using Azure OpenAI service 使用 Azure 时无法切换模型

[KevenLi8888]

Describe the bug The project currently requires the AZURE_URL environment variable in the format like https://{azure-resource-url}/openai/deployments/{deploy-name}, which enforces the inclusion of the deployment name. However, according to the Azure OpenAI documentation [1] and [2], it's not possible to send requests to different models using a URL that specifies the deployment name. The relationship between deployment names and models are 1-to-1 (i.e. models are specified with the deployment name). This is different from the OpenAI API, where the model is specified in the request body.

For instance, consider having two models, gpt-3.5 and gpt-4 from the Azure OpenAI service. They would each have distinct URLs, such as https://{azure-resource-url}/openai/deployments/gpt35 and https://{azure-resource-url}/openai/deployments/gpt4. In the current project implementation, switching between models is accomplished by altering the model field within the request body. However, this approach is not compatible with the Azure OpenAI service, where the deployment name portion of the URL needs to be changed instead.

Relevent Discussions https://github.com/Yidadaa/ChatGPT-Next-Web/pull/3206#issuecomment-1807525917 https://github.com/Yidadaa/ChatGPT-Next-Web/pull/3206#issuecomment-1807529197 https://github.com/Yidadaa/ChatGPT-Next-Web/pull/3260

Expected behavior A potential solution is to include a model name to deployment name mapper in the environment variables. For reference, see https://github.com/stulzq/azure-openai-proxy.

[KevenLi8888]

项目要求在AZURE_URL中提供包含deployment-name的URL，但在 Azure 的概念中，一个deployment-name对应一个模型，并非像 OpenAI 那样在 request body 中指定模型。这导致项目无法按照用户预期切换不同模型。

参见 Azure 文档 [1] [2] https://github.com/Yidadaa/ChatGPT-Next-Web/pull/3206#issuecomment-1807525917 https://github.com/Yidadaa/ChatGPT-Next-Web/pull/3206#issuecomment-1807529197 https://github.com/Yidadaa/ChatGPT-Next-Web/pull/3260

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

The project requires providing a URL containing deployment-name in AZURE_URL, but in the concept of Azure, a deployment-name corresponds to a model, instead of specifying the model in the request body like OpenAI. This resulted in the project being unable to switch between different models as users expected.

See Azure documentation [1] [2] https://github.com/Yidadaa/ChatGPT-Next-Web/pull/3206#issuecomment-1807525917 https://github.com/Yidadaa/ChatGPT-Next-Web/pull/3206#issuecomment-1807529197 https://github.com/Yidadaa/ChatGPT-Next-Web/pull/3260

[H0llyW00dzZ]

项目要求在AZURE_URL中提供包含deployment-name的URL，但在 Azure 的概念中，一个deployment-name对应一个模型，并非像 OpenAI 那样在 request body 中指定模型。这导致项目无法按照用户预期切换不同模型。

参见 Azure 文档 [1] [2] #3206 (comment) #3206 (comment) #3260

yeah I agreed azure is used single model in 1 deployment (example)

anyways model in request body it won't affected (e.g you switch model to gpt-4 in request body like openai service, but deployment only gpt-35 it will force used gpt-35 model)

[H0llyW00dzZ]

update this another issue need external handling complex (k8s aka kubernetes) I guess for azure deployment

[KevenLi8888]

update this another issue need external handling complex (k8s aka kubernetes) I guess for azure deployment

I don't see why we need Kubernetes here. Could you elaborate a bit? I think the key for this issue is to support the way of specifying models when using Azure endpoints, which is different from the OpenAI API.

In my opinion, the simplest way is to supply a deployment-name to model map in the config, and the service would use the deployment-name that corresponds to the user specified model, concat it into the endpoint url, and make request to the Azure service.

[H0llyW00dzZ]

update this another issue need external handling complex (k8s aka kubernetes) I guess for azure deployment

I don't see why we need Kubernetes here. Could you elaborate a bit? I think the key for this issue is to support the way of specifying models when using Azure endpoints, which is different from the OpenAI API.

In my opinion, the simplest way is to supply a deployment-name to model map in the config, and the service would use the deployment-name that corresponds to the user specified model, concat it into the endpoint url, and make request to the Azure service.

you need to build it as own for a external managing multipe complex api for azure

[KevenLi8888]

update this another issue need external handling complex (k8s aka kubernetes) I guess for azure deployment

I don't see why we need Kubernetes here. Could you elaborate a bit? I think the key for this issue is to support the way of specifying models when using Azure endpoints, which is different from the OpenAI API. In my opinion, the simplest way is to supply a deployment-name to model map in the config, and the service would use the deployment-name that corresponds to the user specified model, concat it into the endpoint url, and make request to the Azure service.

you need to build it as own for a external managing multipe complex api for azure

This might be a more complicated way to tackle the problem. Yes there could be multiple deployments created in the Azure service that corresponds to the same type of model, but for the sake of simplicity, on the project level, supporting a one-to-one relationship between deployment-name and model is sufficient for most people.

Azure deployment-name (user defined)	OpenAI model type
my-azure-gpt4	gpt-4
gpt4-32-us-east	gpt-4-32k
gpt-35-turbo	gpt-3.5-turbo
can-be-any-name-user-wants	gpt-3.5-turbo-16k

[H0llyW00dzZ]

update this another issue need external handling complex (k8s aka kubernetes) I guess for azure deployment

I don't see why we need Kubernetes here. Could you elaborate a bit? I think the key for this issue is to support the way of specifying models when using Azure endpoints, which is different from the OpenAI API. In my opinion, the simplest way is to supply a deployment-name to model map in the config, and the service would use the deployment-name that corresponds to the user specified model, concat it into the endpoint url, and make request to the Azure service.

you need to build it as own for a external managing multipe complex api for azure

This might be a more complicated way to tackle the problem. Yes there could be multiple deployments created in the Azure service that corresponds to the same type of model, but for the sake of simplicity, on the project level, supporting a one-to-one relationship between deployment-name and model is sufficient for most people.

Azure deployment-name (user defined) OpenAI model type my-azure-gpt4 gpt-4 gpt4-32-us-east gpt-4-32k gpt-35-turbo gpt-3.5-turbo can-be-any-name-user-wants gpt-3.5-turbo-16k

must refactor in this one to made it support multipe only for azure

[vsare]

I think you can consider the project cf-openai-azure-proxy, which uses cloudflare to convert azure openai to the API that applies to openai, and it also supports multi-model switching, which is how I deployed this site and it works fine!

[lloydzhou]

现在已经支持切换（需要结合CUSTOM_MODELS配置deploy_name）

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Switching is now supported (deploy_name needs to be configured in conjunction with CUSTOM_MODELS)