Getting this backing off problem when I use COPRO

stanfordnlp / dspy

DSPy: The framework for programming—not prompting—foundation models

https://dspy-docs.vercel.app/

MIT License

14.02k stars 1.07k forks source link

Open ArslanS1997 opened 3 weeks ago

ArslanS1997 commented 3 weeks ago

Hi no matter which API I use, groq or OpenAI I get this message repeatedly, backing off after x tries

For groq

From OpenAI using gpt 4o

What does this error precisely mean and anyway to mitigate this?

arnavsinghvi11 commented 2 weeks ago

Hi @ArslanS1997 ,

Try reducing the number of threads so COPRO doesn’t make too many requests simultaneously. This should help avoid triggering the rate limit.