Groq is too fast - Githubissues

jackMort / ChatGPT.nvim

ChatGPT Neovim Plugin: Effortless Natural Language Generation with OpenAI's ChatGPT API

Apache License 2.0

3.56k stars 307 forks source link

Groq is too fast #413

Open Aman9das opened 3 months ago

Aman9das commented 3 months ago

The output from https://console.groq.com gets cut off due to the speed of inference.

anandkumar89 commented 3 months ago

How do you setup groq with this plugin?

Aman9das commented 3 months ago

using environment variables OPENAI_API_HOST and OPENAI_API_KEY

ehsan2003 commented 2 days ago

how you have set model ?

thiswillbeyourgithub commented 2 days ago

Personnaly I ended up making a quick and dirty docker compose to run litellm and highjack request. That allowed me to use Claude sonnet 3.5 as well as online searching models.