search

Future-House / paper-qa

High accuracy RAG for answering questions from scientific documents with citations
Apache License 2.0
6.36k stars 603 forks source link

Malformed message error #419

Open rcinco640 opened 1 month ago

rcinco640 commented 1 month ago

HI. I am getting error messages saying that litellm.exceptions.RateLimitError: litellm.RateLimitError: RateLimitError: OpenAIException - Error code: 429 - {'error': {'message': 'Rate limit reached for gpt-4o in organization org-Uu96uahYiQsvuhVZ23l7Tvjp on tokens per min (TPM): Limit 30000, Used 29204, Requested 1821. Please try again in 2.05s. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}. then there is a repeat of Retrying in 0.0 seconds as it raised MalformedMessageError: Unexpected shape of LiteLLM model response
ModelResponse(id='chatcmpl-A84rdhzaylp70GTIZCfb9RwdkHEKp', choices=[Choices(finish_reason='stop', index=0, message=Message(content='It seems
there is an issue with the paper search tool. I am unable to retrieve papers at the moment. Please try again later or check the query for any
possible issues.', role='assistant', tool_calls=None, function_call=None))], created=1726488685, model='gpt-4o-2024-08-06',
object='chat.completion', system_fingerprint='fp_bbdc8689fd', usage=Usage(completion_tokens=36, prompt_tokens=689, total_tokens=725,
completion_tokens_details={'reasoning_tokens': 0}), service_tier=None).. I have run this several times to no avail. I have not got it to work.

dosubot[bot] commented 1 month ago

Hello, @rcinco640! I'm here to help you with any bugs or questions you have while waiting for a human maintainer.

I found a similar closed issue regarding rate limit errors with the OpenAI API. The suggested solution was to ensure that you have paid for the OpenAI API usage [1].

The rate limits for the OpenAI API are:

The retry logic in the paper-qa library for handling rate limit errors uses the tenacity library. It retries the request under specific conditions, such as server disconnection or certain client response errors, up to 5 times with incremental waiting periods [7].

For the MalformedMessageError, common causes include incorrect message format, invalid tool selection, and incomplete or corrupted messages. The retry logic in the run_aviary_agent function handles this error by retrying the tool selection process up to 5 times [8].

To resolve your issue:

  1. Ensure you have not exceeded the rate limits set by OpenAI.
  2. Verify that your messages are correctly formatted and complete.
  3. Check the implementation of your tools to ensure they are valid and robust.

If the issue persists, you may need to adjust the retry behavior or rate limit handling in your configuration.

To continue talking to Dosu, mention @dosu.

MaximilianHoffmann commented 1 month ago

I am also reaching the rate limits with a simple ask query for a Tier1 OpenAI Usage API - any idea how to fix this?

jamesbraza commented 1 month ago

Hi @MaximilianHoffmann we are on that here: https://github.com/Future-House/paper-qa/issues/377