search

xenodium / chatgpt-shell

ChatGPT and DALL-E Emacs shells + Org babel 🦄 + a shell maker for other providers
https://xenodium.com
GNU General Public License v3.0
805 stars 73 forks source link

Groq's API support #200

Open pramatias opened 6 months ago

pramatias commented 6 months ago

Hello, will there be an option to add groq-shell as an additional option to chatgpt-shell? I would add it myself, but i am not that well versed in elisp for the moment.

From groq's website [1]: " Groq's APIs are designed to be compatible with OpenAI's, with the goal of making it easy to leverage Groq in applications you may have already built. However, there are some nuanced differences where support is not yet available."

I wrote a very simple example to illustrate how easy it is to use Groq's API, in Rust [2]. Groq is so fast that while a streaming token flag can be enabled, it's totally unnecessary. A simple request i did, took 70 ms to complete.

[1] https://console.groq.com/docs/openai [2] https://github.com/pramatias/groq_test

xenodium commented 6 months ago

Thanks for filing.

After initial requests to make the codebase more reusable for other services, I built https://xenodium.com/a-shell-maker in the hope folks using different providers would fill the gap. This didn't quite pan out.

I've been meaning to fill part of this gap myself, but haven'd had a chance to. Need to give it more thought. The example you shared helps. Thank you.

pramatias commented 5 months ago

Hello @xenodium . I used chatgpt-shell but now i've switched to gptel. I like the unobtrusive idea of gptel better.

Project's author karthink managed to integrate groq's api to gptel pretty easily (see gptel's readme). Anyway best of luck to you and your project chatgpt-shell. Cheers!

xenodium commented 5 months ago

Thanks for the follow-up. I'll keep this issue open to track Groq support.

On a somewhat related note, I've been experimenting with a hybrid model to interact with the shell https://xenodium.com/a-chatgpt-shell-compose-ux-experiment