Rate Limit Error with OpenAI API When Using VisionAgent

[moutasemalakkad]

Description:

I am encountering a rate limit error when trying to use the VisionAgent. The error occurs during the initialization of the VisionAgent and seems to be related to the OpenAI API quota. Below are the details of the error and the steps I have taken to troubleshoot the issue.

Please note (I am a Plus member)

Error Details:

INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 429 Too Many Requests"
INFO:openai._base_client:Retrying request to /embeddings in 0.948247 seconds
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 429 Too Many Requests"
INFO:openai._base_client:Retrying request to /embeddings in 1.822015 seconds
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 429 Too Many Requests"
Traceback (most recent call last):
  File "/Users/moutasemhome/Desktop/from vision_agent.py", line 2, in <module>
    agent = VisionAgent(verbosity=2)
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/vision_agent/agent/vision_agent.py", line 429, in __init__
    Sim(T.TOOLS_DF, sim_key="desc")
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/vision_agent/utils/sim.py", line 46, in __init__
    self.df["embs"] = self.df[sim_key].apply(
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/pandas/core/series.py", line 4924, in apply
    ).apply()
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/pandas/core/apply.py", line 1427, in apply
    return self.apply_standard()
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/pandas/core/apply.py", line 1507, in apply_standard
    mapped = obj._map_values(
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/pandas/core/base.py", line 921, in _map_values
    return algorithms.map_array(arr, mapper, na_action=na_action, convert=convert)
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/pandas/core/algorithms.py", line 1743, in map_array
    return lib.map_infer(values, mapper, convert=convert)
  File "lib.pyx", line 2972, in pandas._libs.lib.map_infer
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/vision_agent/utils/sim.py", line 47, in <lambda>
    lambda x: get_embedding(self.client, x, model=self.model)
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/vision_agent/utils/sim.py", line 15, in get_embedding
    return client.embeddings.create(input=[text], model=model).data[0].embedding
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/openai/resources/embeddings.py", line 114, in create
    return self._post(
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/openai/_base_client.py", line 1240, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/openai/_base_client.py", line 921, in request
    return self._request(
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/openai/_base_client.py", line 1005, in _request
    return self._retry_request(
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/openai/_base_client.py", line 1053, in _retry_request
    return self._request(
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/openai/_base_client.py", line 1005, in _retry_request
    return self._retry_request(
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/openai/_base_client.py", line 1053, in _retry_request
    return self._request(
  File "/Users/moutasemhome/vision_agent/lib/python3.10/site-packages/openai/_base_client.py", line 1020, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.RateLimitError: Error code: 429 - {'error': {'message': 'You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.', 'type': 'insufficient_quota', 'param': None, 'code': 'insufficient_quota'}}

Steps Taken for Troubleshooting:

1. Verified the OPENAI_API_KEY is set correctly.
2. Ensured that the API key has sufficient quota.
3. Implemented retries with exponential backoff to handle rate limits.

Code Snippet:

from vision_agent.agent import VisionAgent

agent = VisionAgent(verbosity=2)
code = agent("What percentage of the area of the jar is filled with coffee beans?", media="jar.jpg")

def get_agent_response(agent, query, media):
    retries = 5
    for i in range(retries):
        try:
            return agent(query, media=media)
        except openai.error.RateLimitError as e:
            print(f"Rate limit exceeded, retrying in {2 ** i} seconds...")
            time.sleep(2 ** i)
        except Exception as e:
            print(f"An error occurred: {e}")
            break
    raise Exception("Rate limit exceeded, all retries failed.")

# Initialize VisionAgent
agent = VisionAgent(verbosity=2)

# Query the agent
try:
    response = get_agent_response(agent, "What percentage of the area of the jar is filled with coffee beans?", "jar.jpg")
    print(response)
except Exception as e:
    print(f"Failed to get a response: {e}")

Additional Information:

• OpenAI API usage and limits can be checked at https://platform.openai.com/settings/organization/limits.
• The error seems to be related to the rate limits set by the OpenAI API.

Questions:

1. Is there a way to handle rate limits more effectively within the VisionAgent?
2. Are there any recommended approaches to optimize API calls to avoid hitting rate limits?

Environment:

• Python Version: 3.10
• Operating System: macOS

Thank you for your assistance!

[dillonalaird]

This error happens when you are using the Free Tier for the OpenAI API. The OpenAI API is different than the ChatGPT paid subscription (~$20/month) and you must buy at least $5 of API credits to enable it https://platform.openai.com/docs/guides/rate-limits/usage-tiers?context=tier-free

[sfc-gh-makkad]

Yes, that worked! Thanks Dillon