[Bug]: RateLimitError breaks agent

tobitege commented 1 month ago

Is there an existing issue for the same bug?

[X] I have checked the troubleshooting document at https://docs.all-hands.dev/modules/usage/troubleshooting
[X] I have checked the existing issues.

Describe the bug

Using e.g. OpenRouter, I got a RateLimitError and can't continue afterwards as the agent is "broken" and I need to restart the runtime. This was caught till a recent change to the LLM class and hasn't been addressed yet that the agent stays in an operatable mode.

 Error in agent loop: litellm.RateLimitError: RateLimitError: OpenrouterException -
Traceback (most recent call last):
  File "/home/tobias/.cache/pypoetry/virtualenvs/openhands-ai-Unl_RagG-py3.12/lib/python3.12/site-packages/litellm/llms/OpenAI/openai.py", line 810, in completion
    raise e
  File "/home/tobias/.cache/pypoetry/virtualenvs/openhands-ai-Unl_RagG-py3.12/lib/python3.12/site-packages/litellm/llms/OpenAI/openai.py", line 761, in completion
    return convert_to_model_response_object(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tobias/.cache/pypoetry/virtualenvs/openhands-ai-Unl_RagG-py3.12/lib/python3.12/site-packages/litellm/utils.py", line 5903, in convert_to_model_response_object
    raise raised_exception
Exception

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tobias/.cache/pypoetry/virtualenvs/openhands-ai-Unl_RagG-py3.12/lib/python3.12/site-packages/litellm/main.py", line 2072, in completion
    response = openai_chat_completions.completion(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tobias/.cache/pypoetry/virtualenvs/openhands-ai-Unl_RagG-py3.12/lib/python3.12/site-packages/litellm/llms/OpenAI/openai.py", line 820, in completion
    raise OpenAIError(
litellm.llms.OpenAI.openai.OpenAIError

During handling of the above exception, another exception occurred:

  File "/mnt/d/github/workspace/OpenHands/openhands/controller/agent_controller.py", line 422, in _step
    action = self.agent.step(self.state)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/d/github/workspace/OpenHands/openhands/agenthub/codeact_agent/codeact_agent.py", line 207, in step
    response = self.llm.completion(**params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tobias/.cache/pypoetry/virtualenvs/openhands-ai-Unl_RagG-py3.12/lib/python3.12/site-packages/tenacity/__init__.py", line 336, in wrapped_f
    return copy(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/tobias/.cache/pypoetry/virtualenvs/openhands-ai-Unl_RagG-py3.12/lib/python3.12/site-packages/tenacity/__init__.py", line 475, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tobias/.cache/pypoetry/virtualenvs/openhands-ai-Unl_RagG-py3.12/lib/python3.12/site-packages/tenacity/__init__.py", line 376, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/home/tobias/.cache/pypoetry/virtualenvs/openhands-ai-Unl_RagG-py3.12/lib/python3.12/site-packages/tenacity/__init__.py", line 418, in exc_check
    raise retry_exc.reraise()
          ^^^^^^^^^^^^^^^^^^^
  File "/home/tobias/.cache/pypoetry/virtualenvs/openhands-ai-Unl_RagG-py3.12/lib/python3.12/site-packages/tenacity/__init__.py", line 185, in reraise
    raise self.last_attempt.result()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/tobias/.cache/pypoetry/virtualenvs/openhands-ai-Unl_RagG-py3.12/lib/python3.12/site-packages/tenacity/__init__.py", line 478, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/mnt/d/github/workspace/OpenHands/openhands/llm/llm.py", line 191, in wrapper
    resp: ModelResponse = completion_unwrapped(*args, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tobias/.cache/pypoetry/virtualenvs/openhands-ai-Unl_RagG-py3.12/lib/python3.12/site-packages/litellm/utils.py", line 1071, in wrapper
    raise e
  File "/home/tobias/.cache/pypoetry/virtualenvs/openhands-ai-Unl_RagG-py3.12/lib/python3.12/site-packages/litellm/utils.py", line 959, in wrapper
    result = original_function(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tobias/.cache/pypoetry/virtualenvs/openhands-ai-Unl_RagG-py3.12/lib/python3.12/site-packages/litellm/main.py", line 2921, in completion
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/home/tobias/.cache/pypoetry/virtualenvs/openhands-ai-Unl_RagG-py3.12/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2131, in exception_type
    raise e
  File "/home/tobias/.cache/pypoetry/virtualenvs/openhands-ai-Unl_RagG-py3.12/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2038, in exception_type
    raise RateLimitError(
litellm.exceptions.RateLimitError: litellm.RateLimitError: RateLimitError: OpenrouterException -

Current OpenHands version

0.9.8

Installation and Configuration

from source

Model and Agent

No response

Operating System

No response

Reproduction Steps

No response

Logs, Errors, Screenshots, and Additional Context

No response

github-actions[bot] commented 1 week ago

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

enyst commented 1 week ago

This may have been handled by the recent error handling refactoring. If the agent encounters an exception it cannot get out of, the UI will display a notice, and the user should be able to continue.

All-Hands-AI / OpenHands