LLM RATE LIMIT WITH GROQ MODELS

[Bless50]

Describe the bug warnings.warn( Error in generate_report: Error code: 413 - {'error': {'message': 'Request too large for model llama-3.1-70b-versatile in organization org_01hw8f0aa0ekmrn70ffm2zkdv1 on tokens per minute (TPM): Limit 6000, Requested 14368, please reduce your message size and try again. Visit https://console.groq.com/docs/rate-limits for more information.', 'type': 'tokens', 'code': 'rate_limit_exceeded'}}

To Reproduce Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [windows 10 pro]
• Browser [e.g. chrome]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context Add any other context about the problem here.

[ElishaKay]

Welcome @Bless50,

Try changing your LLM config variables:

FAST_LLM="ollama:qwen2:1.5b"
SMART_LLM="ollama:qwen2:1.5b"
EMBEDDING="ollama:all-minilm:22m"

OR: llama3.1

Looks from the error message that the rate limit on the API is shared across anyone else in your organization using the same model