Azure: How to specify both model and deployment?

[ivantheterrible]

Issue

TLDR: With Azure OpenAI, the deployment name is a different thing to the model name. Aider presumably needs to know both. I'm not clear how you're supposed to specify both the model and the deployment?

Longer description:

With Azure OpenAI, the --model parameter seems to be where you specify azure deployment name. For example, if I have a deployment is called TEST, I might invoke aider thus:

aider --no-auto-commits --model azure/TEST

it will then work, but it says::

Model azure/TEST: Unknown context window size and costs, using sane defaults.

Let's say the TEST deployment is in fact a gpt-4o model. You can invoke aider like this:

aider --no-auto-commits --model azure/gpt-4o

But then it doesn't actually know the deployment name, so you get an error when making the first request:

Unexpected error: litellm.NotFoundError: AzureException NotFoundError - Error code: 404 - {'error': {'code': 'DeploymentNotFound', 'message': 'The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.'}}

Version and model info

Aider v0.53.0 Azure OpenAI gpt-4o

[paul-gauthier]

Thanks for trying aider and filing this issue. These docs may be helpful:

https://aider.chat/docs/llms/azure.html https://aider.chat/docs/config/adv-model-settings.html

You may need to manually configure all the settings and metadata for your Azure deployment.

[ivantheterrible]

@paul-gauthier thank you. It sounds like if I could make a model-settings-file that would actually relate to my model, that might help. How can I found how to create the correct model-settings-file for (e.g.) gpt-4o ?

[sharad4815]

@ivantheterrible I had the similar issue, the thing is use deployment name regardless whichever model you are using, Because the model is needed to be set-up in Azure portal. I had the same issue, if you used model name here, it will not work.

[ivantheterrible]

@sharad4815 yes, to get it to work I do need to use the deployment name after '--model'. But aider then complains

Model azure/TEST: Unknown context window size and costs, using sane defaults.

So I'd like to know how to tell it about the model (given that --model is actually needed to define the deployment name)

[paul-gauthier]

You will need a model metadata and model settings file if you want to fully configure an unknown model for aider.

See litellm's model_prices_and_context_window.json file for model metadata examples.

You can look at the ModelSettings class in models.py file for details about all of the model setting that aider supports. That file also contains the settings for many popular models.

[paul-gauthier]

I'm going to close this issue for now, but feel free to add a comment here and I will re-open. Or feel free to file a new issue any time.