search

zylon-ai / private-gpt

Interact with your documents using the power of GPT, 100% privately, no data leaks
https://docs.privategpt.dev
Apache License 2.0
52.89k stars 7.12k forks source link

Groq API in PrivateGPT #1945

Open psmukhopadhyay opened 1 month ago

psmukhopadhyay commented 1 month ago

Hello

I am a beginner-level user of PrivateGPT and set it for 'local' with LLM as mistral-7b-instruct-v0.2.Q4_K_M.gguf.

Please advise me how to add Groq (OpenAI compatible LLM service - https://console.groq.com/docs/quickstart) in PrivateGPT. I already have API key and other ID for Groq. Should I follow the included azopenai as model?

Regards

KenFlack commented 1 month ago

I am in the same boat... Will let you know if I discover the solution.

psmukhopadhyay commented 1 month ago

Thanks Ken :)