search

joaomdmoura / crewAI

Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.
https://crewai.com
MIT License
16.92k stars 2.29k forks source link

crashes on first occurrence of OpenAI rate limit #377

Open aliensouls opened 3 months ago

aliensouls commented 3 months ago

it looks like the agents cannot retry or retry with exponential backoff intervals, when receiving rate limiting from openai api. the tokens per minute can be reached easily, if many agents work with large contexts, so the agents should simply wait and retry - but at the moment it halts the crew and never resumes. would it be possible to add a timeout and make it retry, instead of halting all operation when throttling from openai or any other api gets received?

example of implementation: https://github.com/openai/openai-cookbook/blob/main/examples/api_request_parallel_processor.py#L123

should I submit a PR? I'm new to this project, but if you could point me to the correct place where this processing takes place, I might be able to help.

aliensouls commented 3 months ago

I see something like that already implemented class RPMController(BaseModel): but doesn't seem to work in my case.. where is the limit specified? all limits are different per each model, and also every account tier, so my current low tier also makes the limits lower than others - is it hardcoded or received from openai specifically about my account? in any case, it doesn't work (I crash on "tokens per minute limit reached, please wait" error from openai api, during one of the tasks, and it goes kaboom 💥