Cannot setup an Azure OpenAI model

continuedev / continue

⏩ Continue is the leading open-source AI code assistant. You can connect any models and any context to build custom autocomplete and chat experiences inside VS Code and JetBrains

https://docs.continue.dev/

Apache License 2.0

16k stars 1.22k forks source link

Cannot setup an Azure OpenAI model #1803

Open kedar-1 opened 1 month ago

kedar-1 commented 1 month ago

Before submitting your bug report

[ ] I believe this is a bug. I'll try to join the Continue Discord for questions
[ ] I'm not able to find an open issue that reports the same bug
[ ] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS:Windows
- Continue:last
- IDE: VSCode
- Model: gpt-4o
- config.json:

Description

How to setup an Azure Open AI model?

What is Engine? Where I can grab this value in Azure portal/AI Studio?

What is API Base? Where I can grab this value in Azure portal/AI studio?

Please create a documentation not only what must be inserted into config file but WHERE I can find the required value.

Thanks

To reproduce

Open a Continue.dev.
Click on + to Add model.
Select an Azure

Log output

No response

geekloper commented 1 month ago

I agree that the documentation for the Azure OpenAI service could be clearer. Here's how I configured mine:

Configuration in config.json

{
  "models": [{
    "title": "Azure OpenAI",
    "provider": "azure",
    "model": "<model>",
    "engine": "<deployment_id>",
    "apiBase": "<azure_open_ai_endpoint>",
    "apiType": "azure",
    "apiKey": "<azure_open_ai_key>"
  }...

Explanation of Configuration Parameters

<model>: You can find this in the deployment section of Azure OpenAI Studio (e.g., gpt35turbo).
<deployment_id>: This is the name you chose for the model in Azure OpenAI Studio, also located in the deployment section.
<azure_open_ai_endpoint> & <azure_open_ai_key>: This can be found under the Azure OpenAI resource in the Azure portal:
- Navigate to Resource Management
- Select Keys and Endpoint

ChristophRaab commented 1 month ago

I agree with @geekloper. Thats also how i configured it. However, i looked up the current ModelDescription-Schema but i think the "engine", "apiVersion" and "apiType" may missing. If i follow the model description, i am not able to come up with a working azure models.

See: https://github.com/continuedev/continue/blob/0ef443cafc00eb44223bb34c81e52c3b1910eb3e/core/index.d.ts#L714-L726

kedar-1 commented 1 month ago

Thanks @geekloper. With your help I was able to make a Continue.dev extension to work with GPT-4o model. I'll try to create a PR as the documentation on project site its not user friendly.

BTW: Setting up TabAutocomplettion to llama3.1 running on ollama local I have a nice, smart and cheap Copilot :-)