entropy-research / Devon

Devon: An open-source pair programmer
GNU Affero General Public License v3.0
3.28k stars 264 forks source link

Devon UI stuck on rate limit reached: automatically retrying in 1 minute infinitely #85

Closed cbrane closed 4 months ago

cbrane commented 5 months ago

Hi, I was running into this issue earlier, and with a restart it fixed it, but now it is just in an infinite loop of rate limit reached, even restarting 3-4 times and the issue is not being solved. I tried to both continue my old conversation and start a new one, and the same issue is happening over and over again. Here are the logs, they're identical every time the issue happens on an old or new convo and won't go away:

Server: INFO: 127.0.0.1:57277 - "GET /sessions/UI/state HTTP/1.1" 200 OK

Server: INFO: 127.0.0.1:57277 - "GET /sessions/UI/state HTTP/1.1" 200 OK

Server: event {'type': 'ModelRequest', 'content': '', 'producer': 'Devon', 'consumer': 'Devon'} get_engine <sqlalchemy.ext.asyncio.engine.AsyncEngine object at 0x107560750>

Saving data for: UI

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new LiteLLM.Info: If you need to debug this error, use `litellm.set_verbose=True'.

event {'type': 'EnvironmentRequest', 'content': 'pwd', 'producer': 'tool', 'consumer': 'local'} event {'type': 'EnvironmentResponse', 'content': 'my directory that i chose here (replacing for privacy)', 'producer': 'local', 'consumer': 'tool'} event {'type': 'EnvironmentRequest', 'content': 'pwd', 'producer': 'tool', 'consumer': 'local'} event {'type': 'EnvironmentResponse', 'content': 'my directory that i chose here(replacing for privacy)', 'producer': 'local', 'consumer': 'tool'} event {'type': 'RateLimit', 'content': '', 'producer': 'Devon', 'consumer': 'none'} INFO: 127.0.0.1:57277 - "GET /sessions/UI/state HTTP/1.1" 200 OK

Server: INFO: 127.0.0.1:57277 - "GET /sessions/UI/state HTTP/1.1" 200 OK

Server: INFO: 127.0.0.1:57277 - "GET /sessions/UI/state HTTP/1.1" 200 OK

Server: INFO: 127.0.0.1:57277 - "GET /sessions/UI/state HTTP/1.1" 200 OK

Server: INFO: 127.0.0.1:57277 - "GET /sessions/UI/state HTTP/1.1" 200 OK

Server: INFO: 127.0.0.1:57277 - "GET /sessions/UI/state HTTP/1.1" 200 OK

99Gens commented 4 months ago

Claude token limit is 1mm in and out combined per 24hours resetting at 5pm PST each day. When you see this loop it means Devon is getting rate limited. For Claude 3 models, the limits are 5 requests per minute (RPM) and between 10,000 to 25,000 tokens per minute (TPM), depending on the specific model. My suggestion to the Devon team is to reprogram this message to be clearer. Something like "You have exceeded the tokens per 60 seconds limit, taking a breath and trying again in 1 min" and "You are maxed out of daily tokens on Claude". OpenAI has rate limits as well, so a lot of users swap in a GPT4o key as soon as they hit their daily rate on Claude. However, Claude is a better engineer and OpenAI is more expensive. The approach which I've found generally most useful in not burning out all my tokens is to create tests, install a micro-agent. Learn about that here: https://x.com/steve8708/status/1804295594879652197 and then it will seriously reduce the tokens being sent back and forth. I think the wide developer community is patiently awaiting the day Claude's Artifacts feature is available via the API. DEVON TEAM: Let's have a feature where I can keep my chat going and swap my Claude Key? yes? That would be very cool. Right now, when I use a second key, I have to reboot my entire session. I burn significant % of my daily tokens waiting for Devon to relearn everything about my project. :)