Add support for fill-in-the-middle code completion (codestral)

[timinou]

Hello!

Mistral has recently released Codestral, which supports fill-in-the-middle. In gptel, this could be implemented with a custom function that sends a "suffix" string, e.g. with a certain number of lines. If we wanted to get fancy, it could include a certain stop determined by a stopword, or even a treesitter-determined stop.

My limited trials with codestral have shown it to be quite fast, so I'm imagining a command in my config that could generate the code with a direct shortcut (e.g. M-C-i), and the suggestion would show up as a grey suggestion à la company.

My knowledge of emacs lisp is limited, but I am happy to contribute to this if you give me a couple pointers.

Currently, I have Codestral configured like so:

 (gptel-make-openai "Mistral"
  :host "api.mistral.ai"
  :key (getenv "MISTRAL_API_KEY")
  :stream t
  :models '("codestral-latest"))

Thank you!

[karthink]

There isn't an easy way to add support for a bespoke API for one single model right now. This requires the ability to add metadata per model to gptel (instead of per provider/backend, like "Ollama" or "OpenAI"). This is planned but I haven't worked on it yet.

Alternatively, if Codestral's FIM follows a semi-standard API used by other FIM-supported models and providers, it might be worth adding a special case to gptel.

[vdm]

https://gitlab.com/daanturo/starhugger.el

[deepanshu44]

I have developed somewhat a hacky solution using curl, that should make things work. This is completely elisp based and doesn't require any package. Make sure to add your $MISTRAL_API_KEY.

(defun gpt-test ()
  (interactive "")
  (replace-regexp-in-string
   ;; emacs will add a newline when '\n' is encountered
   "\\\\n" "\n"
   (replace-regexp-in-string
    ;; output cleanup
    ;; remove everything before "content" property and after "tools_call" property
    ;; only the required user code remains
    "\\(\",\"tool_calls.*\\|.*content\":\"\\)" ""
    (shell-command-to-string
     (concat "curl -s \
     --location \"https://codestral.mistral.ai/v1/fim/completions\"\
     --header 'Content-Type: application/json'\
     --header 'Accept: application/json'\
     --header \"Authorization: Bearer $MISTRAL_API_KEY\"\
     --data '\{\"model\":\"codestral-latest\",
               \"prompt\":\"" (string-join (split-string (buffer-substring-no-properties (point-min) (point)) "\n") "\\n") "\",
               \"suffix\": \""  "\",
               \"max_tokens\": 164,
               \"temperature\": 0
\}'")))))