(Doom Emacs) Help with using gptel

[pramatias]

Hello, i used chatgpt-shell lately, but now that i looked into gptel i like the simplicity of it a lot. I was wondering if you could add support for Groq in addition to all the other LLM's.

From groq's website [1]: " Groq's APIs are designed to be compatible with OpenAI's, with the goal of making it easy to leverage Groq in applications you may have already built. However, there are some nuanced differences where support is not yet available."

I wrote a very simple example to illustrate how easy it is to use Groq's API, in Rust [2]. Groq is so fast that while a streaming token flag can be enabled, it's totally unnecessary. A simple request i did, took 70 ms to complete for almost a page of text.

I mean, it should be fairly easy to integrate it into the existing code. I am looking into the source to see if i can manage it myself.

[1] https://console.groq.com/docs/openai [2] https://github.com/pramatias/groq_test

[karthink]

Sure, could you try

(gptel-make-openai "Groq"
  :host "api.groq.com"
  :endpoint "/openai/v1/chat/completions"
  :stream nil
  :key "your-groq-api-key"
  :models '("mixtral-8x7b-32768"))

and let me know if that works? If it does, also try it with :stream t.

[pramatias]

Thanks for the reply @karthink.

It doesn't work. I even put my OpenAi key, and parameters like that

(gptel-make-openai "Openai" :host "api.openai.com" :endpoint "/v1/chat/completions" :stream nil :key "***" :models '("gpt-3.5-turbo"))

it still doesn't work. I put it in my init.el file, is it supposed to be somewhere else? I put it in init.el and doom/reload after that.

Oh i forgot... The error is: Incorrect API key provided: emacs.d/*****pos/ You can find your API key at platform.openai.com/account/api-keys.

[karthink]

Did you select this backend (groq or openai) before using gptel?

If you didn't do that and don't understand what that means, please try gptel with

(setopt gptel-backend
        (gptel-make-openai "Groq"
          :host "api.groq.com"
          :endpoint "/openai/v1/chat/completions"
          :stream nil
          :key "your-groq-api-key"
          :models '("mixtral-8x7b-32768")))

👆🏼 To use Groq.

To use ChatGPT you don't need to define a backend, you can just set the variable gptel-api-key (see its documentation) instead.

[pramatias]

Thanks for the patience @karthink . I am not a very experienced emacs user. I used chatgpt-shell before, but i like the unobtrusive approach of gptel. I try to run gptel for the first time today. Gptel indeed when i close emacs and open it again, it asks for a key.

I put into emacs.d/lisp/init.el the configuration

(gptel-make-openai "Groq" :host "api.groq.com" :endpoint "/openai/v1/chat/completions" :stream nil :key "your-groq-api-key" :models '("mixtral-8x7b-32768"))

this alone, or with the setopts before that, it never makes a difference. It always asks me for a key when i close emacs and open it again, but not when i reload/doom.

I put the same configuration to .emacs.d/lisp/lib/config.el alongside

use-package! gptel :config (setq! gptel-api-key "key")) no difference again.

Indeed in the home project directory inside README.md, it says something about gptel_backend. I don't know what that means.

[karthink]

I don't know how the init files are structured in Doom emacs, so I can't help you there.

1. In emacs, run the setopt version of the command. Be sure to substitute your actual key for "your-groq-api-key".
2. Run M-x gptel. It should not ask you for a key
3. In the new buffer, call M-x gptel-menu. This menu should show you the active backend and model. It should say "Groq:mixtral..."

If the above is correct, try typing in something and running M-x gptel-send.

On Fri, Mar 15, 2024, 8:37 PM kostas pramatias @.***> wrote:

Thanks for the patience @karthink https://github.com/karthink . I am not a very experienced emacs user. I used chatgpt-shell before, but i like the unobtrusive approach of gptel. I try to run gptel for the first time today. Gptel indeed when i close emacs and open it again, it asks for a key.

I put into emacs.d/lisp/init.el the configuration

(gptel-make-openai "Groq" :host "api.groq.com" :endpoint "/openai/v1/chat/completions" :stream nil :key "your-groq-api-key" :models '("mixtral-8x7b-32768"))

this alone, or with the setopts before that, it never makes a difference. It always asks me for a key when i close emacs and open it again, but not when i reload/doom.

I put the same configuration to .emacs.d/lisp/lib/config.el alongside

use-package! gptel :config (setq! gptel-api-key "key")) no difference again.

Indeed in the home project directory inside README.md, it says something about gptel_backend. I don't know what that means.

— Reply to this email directly, view it on GitHub https://github.com/karthink/gptel/issues/257#issuecomment-2001451873, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACBVOLEFI4U2I3I2SWWSBVTYYO47ZAVCNFSM6AAAAABEY4IHW2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBRGQ2TCOBXGM . You are receiving this because you were mentioned.Message ID: @.***>

[pramatias]

I evaluated the setopts gptel-backend command on the buffer, and now gptel's minibuffer mentions mixtral. Before that it didn't mention mixtral at all, it always pointed to gpt-3.5. Strangely, it ignores this command from the config.el and init.el files, but it runs fine in the scratch buffer.

The thing is, it cannot save the key for some reason. I ran (setq! gptel-api-key "key"), i ran it inside the setopts gptel-backend, it always fails to save the key, and it doesn't ask me again. Once i run the setopts gptel-backend with the key inside, it doesn't ask again, but it also fails to use it.

[karthink]

@pramatias I'm quite confused about the state of your Emacs.

If you run M-x eval-expression (gptel-backend-key gptel-backend) in the above buffer, what does it return?

[pramatias]

Well, when i close and open emacs, gptel asks for an OpenAI key. I give it the key, and it works fine. Then i close it, open it again, and instead of giving it a key, i eval-expression (setq! gptel-api-key "key") and it works fine again. It connects in both cases to gpt-3.5-turbo no problem, as the screenshot shows [1].

Interestingly, it ignores any api-key configuration in init.el, or config.el like README.md mentions ` (use-package! gptel :config (setq! gptel-api-key "your key"))

` No matter what api-key i put there, i always asks me for one.

---

Next step i eval-expression the gptel-backend with groq arguments. Gptel's ui changes as the screenshot shows [2]. Then it fails because the api-key is not correct. I setq! the correct api-key, it still doesn't work.

(setopt gptel-backend (gptel-make-openai "Groq" :host "api.groq.com" :endpoint "/openai/v1/chat/completions" :stream nil :key "key" :models '("mixtral-8x7b-32768")))

Then i try to run eval-expression the gptel-backend with OpenAI arguments as such (setopt gptel-backend (gptel-make-openai "OpenAI" :host "api.openai.com" :endpoint "/v1/chat/completions" :stream nil :key "key" :models '("gpt-3.5-turbo")) By trying to run a new prompt the minibuffer outputs "querying Groq" and it fails.

And that's it. I cannot run gptel again, until i close emacs and open it again.

---

When i run Emacs for the first time, i connect to gpt-3.5-turbo and it works fine, (gptel-backend-key gptel-backend) outputs gptel-api-key.

After i execute setopt gptel-backend Groq and it stops working, (gptel-backend-key gptel-backend) always outputs the Groq's api-key, the correct one.

[1] [2]

[karthink]

Between Doom emacs' unfamiliar settings and macros (I don't know what setq! does), buffer local variables and untested backends like groq, there are too many things going on here for me to help.

Since you are using gptel for the first time, I suggest forgetting about groq for the time being. We'll come back to it after you can use gptel without trouble.

Remove all gptel-related configuration from your init files.

Next, install gptel (however you do it in Doom).

In your init file (I don't know the right one, in regular emacs it's just init.el), add

(use-package gptel
  :config
  (setq gptel-api-key "your-openai-api-key"))

That's it. Now restart Emacs.

In Emacs, now run M-x gptel. It should not ask you for anything. Let me know if ChatGPT works as expected in the gptel buffer.

Also please read the README carefully.

[pramatias]

Ok, i will test it after some hours. Thanks for the help @karthink .

Just to mention, that gptel works fine on Doom Emacs, except that little wrinkle with the api-keys in config.el which do not load by themselves. Even this i am not sure, i will test it to find out. It is only when i try to switch backends when everything stops working suddenly. I might find a way to hack it myself. Cheers!

[pramatias]

Gptel works in Doom Emacs, the documentation in README is correct, the structopt command @karthink wrote is correct as well.

I messed up the keys somehow, the groq keys got downcased for some reason, and they didn't work.

here is the final configuration i put into my config file (.emacs.d/.local/cache/custom.el) if someone is interested.

(defun read-bash-variable (variable-name)
  "Reads a Bash variable and returns its value."
  (let ((output (shell-command-to-string (format "echo $%s" variable-name))))
    (string-trim-right output)))

(use-package! gptel
 :config
   (setq! gptel-api-key (read-bash-variable "OPENAI_API_KEY")))

(setopt gptel-backend
        (gptel-make-openai "Groq"
          :host "api.groq.com"
          :endpoint "/openai/v1/chat/completions"
          :stream nil
          :key (read-bash-variable "GROQ_API_KEY")
          :models '("mixtral-8x7b-32768")))

Maybe i ask a followup question if i may, chatGPT has 4K context window, while mixtral has 32K. Is it possible to extend the context in gptel? Not a big deal, but it would be nice! Thanks for the help anyway!

[karthink]

I messed up the keys somehow, the groq keys got downcased for some reason, and they didn't work.

Okay, then I assume Groq works as expected now?

[pramatias]

Yes everything works great!

One thing i haven't figured out yet, is how to switch back to GPT, which i very rarely use.

I put the setopts gptel-backend to my config file, and it loads Groq by default. When i try to run gptel-menu, it doesn't switch to anything else, but it is not that big of a deal. I may try to setup setopts gptel-backend for GPT and Groq, and switch between them using a key binding. But in reality, maybe i haven't figured out how to use gptel-menu yet.

[karthink]

When i try to run gptel-menu, it doesn't switch to anything else

You'll have to provide some screenshots since I can't tell what you're having trouble with. Or you could go the keybinding route too, yes.