RateLimitError: Exceeded OpenAI API Quota Leading to LLMProviderError Despite 0% Usage on Apple M2 Pro

[frieda-huang]

Hi there!

I keep getting the following errors when trying out any of the Skyvern Templates. Despite my OpenAI API usage being at 0%, I’m encountering a RateLimitError which subsequently causes an LLMProviderError when using the OPENAI_GPT4_TURBO LLM provider. I am running this on an Apple M2 Pro.

 File "/Users/friedahuang/Library/Caches/pypoetry/virtualenvs/skyvern-KFgdp60p-py3.11/lib/python3.11/site-packages/litellm/utils.py", line 7779, in exception_type
    raise RateLimitError(
litellm.exceptions.RateLimitError: OpenAIException - Error code: 429 - {'error': {'message': 'You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.', 'type': 'insufficient_quota', 'param': None, 'code': 'insufficient_quota'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/friedahuang/Documents/skyvern/skyvern/forge/agent.py", line 520, in agent_step
    json_response = await app.LLM_API_HANDLER(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/friedahuang/Documents/skyvern/skyvern/forge/sdk/api/llm/api_handler_factory.py", line 204, in llm_api_handler
    raise LLMProviderError(llm_key) from e
skyvern.forge.sdk.api.llm.exceptions.LLMProviderError: Error while using LLMProvider OPENAI_GPT4_TURBO

Steps to Reproduce

1. Follow Quickstart instructions and run on http://localhost:8080.
2. Attempt to use any of the Skyvern Templates with the OpenAI API.
3. Observe the rate limit being exceeded.
4. Notice the resulting RateLimitError and the subsequent LLMProviderError.

[suchintan]

@frieda-huang we highly recommend using GPT4O if your can

[ykeremy]

Hey @frieda-huang, would you mind checking your openai account tier?

For example, for Tier 1, gpt4 has 30k tokens per minute limit which Skyvern can exceed depending on the webpage you're visiting.

You can try to increase your account tier, here are the tier definitions:

[frieda-huang]

Hey @frieda-huang, would you mind checking your openai account tier?

For example, for Tier 1, gpt4 has 30k tokens per minute limit which Skyvern can exceed depending on the webpage you're visiting.

You can try to increase your account tier, here are the tier definitions:

Thank you for the prompt response! It looks like I'm already on tier 2.

[suchintan]

@frieda-huang could you give GPT_4O a try (instead of GPT-4-Turbo)?

[frieda-huang]

It works now but flashes up the content for a split second whenever some action is taken. It seems to get stuck when navigating the Geico site. Screen Recording 2024-07-10 at 9.10.42 AM.mov https://drive.google.com/file/d/1h3DDL3aeApvaTyu0t7HriQxcy6v9g9JS/view?usp=drive_web

On Wed, Jul 10, 2024 at 8:17 AM Suchintan @.***> wrote:

@frieda-huang https://github.com/frieda-huang could you give GPT_4O a try (instead of GPT-4-Turbo)?

— Reply to this email directly, view it on GitHub https://github.com/Skyvern-AI/skyvern/issues/546#issuecomment-2220787590, or unsubscribe https://github.com/notifications/unsubscribe-auth/A5VHN6A3VJF2ZTOCFFEWBNDZLVGANAVCNFSM6AAAAABKKNEEB6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRQG44DONJZGA . You are receiving this because you were mentioned.Message ID: @.***>

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.