search

microsoft / chat-copilot

MIT License
1.94k stars 663 forks source link

Accommodate Multiple Endpoints and Keys #315

Closed dasheffie closed 8 months ago

dasheffie commented 10 months ago

Is there a way to accommodate multiple endpoints and multiple keys for AzureOpenAI? Microsoft gave us different locations (EASTUS and EASTUS2) for our embedding model than our GPT-3 and GPT-4 access without the option to have all 3 models in one zone. This means that we have a different endpoint and key for the embedding model than our GPT-4 and GPT-3 model access. Is there a way to accommodate a different endpoint and key for each model in a webapi\.env file for docker compose? It would also be nice if this same thing were enabled for webapi\appsettings.json

Our current webapi\.env file looks like this (using fake values)...

Authorization__AzureAd__Scopes=access_as_user
Kestrel__Endpoints__Http__Url=http://0.0.0.0:8080
MemoryStore__Type=qdrant
AIService__Type=AzureOpenAI
AIService__Endpoint=https://oai-dev.azure.com/
AIService__Key=523f002816a1e3r39
AIService__Models__Completion=ai-gpt35turbo-dev
AIService__Models__Planner=ai-gpt4-dev
AIService__Models__Embedding=ai-txtada002-dev

The request is that we would be able to specify different endpoints and keys like this (using fake values)...

Authorization__AzureAd__Scopes=access_as_user
Kestrel__Endpoints__Http__Url=http://0.0.0.0:8080
MemoryStore__Type=qdrant
AIService__Type=AzureOpenAI
AIService__Completion__Endpoint=https://oai-gpt-dev.azure.com/
AIService__Planner__Endpoint=https://oai-gpt-dev.azure.com/
AIService__Embedding__Endpoint=https://oai-dev.azure.com/
AIService__Completion__Key=523f002816a1e3r39
AIService__Planner__Key=523f002816a1e3r39
AIService__Embedding__Key=398jfs0j3092j343r3
AIService__Models__Completion=ai-gpt35turbo-dev
AIService__Models__Planner=ai-gpt4-dev
AIService__Models__Embedding=ai-txtada002-dev
glahaye commented 8 months ago

@dasheffie Chat Copilot now can take different endpoints for completion and embeddings.

In appsetting.json, look for AzureOpenAIText : Endpoint and AzureOpenAIEmbedding : Endpoint