[bug] Codestral update

[guyomes]

Issue

Using mistral/codestral-latest leads to this bug: Unexpected error: litellm.AuthenticationError: AuthenticationError: MistralException - Error code: 401 - {'message': 'Unauthorized', ...}

The codestral api has changed and the codestral keys are separate from the main mistral keys: https://docs.mistral.ai/capabilities/code_generation/

LiteLLM has updated it: https://litellm.vercel.app/docs/providers/codestral

The new model name in LiteLLM is "codestral/codestral-latest" and the key is sotred in "CODESTRAL_API_KEY".

Moreover, in my experience, codestral often abbreviates big chunks of code with "// Rest of the code...". Is there a way maybe to change the prompt to avoid this behavior?

Thank you for this great tool! It really changed my way to think about AI augmented programming.

Version and model info

Aider: v0.39.0 Model: mistral/codestral-latest --> should be changed to codestral/codetral-latest

[paul-gauthier]

Thanks for trying aider and filing this issue.

I'm not aware that aider has any hard coded config for codestral. Whatever litellm supports should work with aider.

[guyomes]

I just upgraded to v.040.0 and it works now, thanks. I still get warning though and indeed, it seems that the codestral variables are not updated on litellm: https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json

I'll update the bug on the litellm project.

> aider --model codestral/codestral-latest --message "test"
Model codestral/codestral-latest: Unknown model, context window size and token costs unavailable.
Did you mean one of these?
- mistral/codestral-latest
https://aider.chat/docs/llms/warnings.html
Aider v0.40.0
Model: codestral/codestral-latest with whole edit format
Git repo: .git with 9 files
Repo-map: disabled

[paul-gauthier]

I'm going to close this issue for now, but feel free to add a comment here and I will re-open or file a new issue any time.

[randoentity]

@guyomes have you found a fix for the "rest of code" response? Codestral seems to be getting it from the prompt, where it should explicitly avoid doing that. Maybe putting it lower in the stack would help?

Edit: so far the following works for me: Append to user prompt: Write the entire code. Do not replace code with comments.
Edit: but it doesn't fix it consistently. Rotating in "WRITE THE WHOLE FILE!!!" ... seems to work! I think I'll have a look at the retry mechanism to try different prompts automatically. Or I'll wait for a new model.