run-llama / LlamaIndexTS

LlamaIndex in TypeScript
https://ts.llamaindex.ai
MIT License
1.59k stars 312 forks source link

Groq is not working anymore #860

Open bharath-naik opened 1 month ago

bharath-naik commented 1 month ago

I've been using Groq for a while, and recently I've encountered an issue where there is no response. Additionally, if I attempt to access the endpoint too frequently, it indicates that the limit has been exceeded. Has there been any recent change in Groq?

activeLlm = new Groq({
          apiKey: process.env.GROQ_API_KEY,
          model:"gemma-7b-it"
        });
marcusschiesser commented 1 month ago

Just tested examples/groq.ts - works nicely with version 0.3.13.

Also with your model:

Settings.llm = new Groq({
  apiKey: process.env.GROQ_API_KEY,
  model: "gemma-7b-it",
});

If you get a "limit has been exceeded" error this sounds like rate limiting from Groq to me - maybe you need a paid version?