search

getomni-ai / zerox

PDF to Markdown with vision models
https://getomni.ai/ocr-demo
MIT License
6.58k stars 358 forks source link

Handle ratelimit and retry #87

Open ccorcos opened 2 weeks ago

ccorcos commented 2 weeks ago

I'm trying with a big pdf and getting a ratelimit error:

    data: {
      error: {
        message: 'Rate limit reached for gpt-4o-mini in organization org-zt8czUs8Thn5aBD4n2Jf1kve on tokens per min (TPM): Limit 200000, Used 198414, Requested 2829. Please try again in 372ms. Visit https://platform.openai.com/account/rate-limits to learn more.',
        type: 'tokens',
        param: null,
        code: 'rate_limit_exceeded'
      }
    }
  },
  status: 429
}

It would be nice if (1) this library would handle those rate limits and (2) it would return intermediate results or something so that I can save results of past pages and keep going when thigns crash or error like this.

Thanks for the help!

tylermaran commented 2 weeks ago

Hey @ccorcos. Good suggestion.

I'd have to think through the ideal response object here. It would need to return some sort of error code as well to let you know that it was giving back a partial response.