Open aa755 opened 4 months ago
Not sure what to handle. The rate limit means your account is running out of quota. 🤔 Retry wouldn't help in these circumstances.
Here is an example response I was getting from the openai-chat
function in this repo:
((error (message . Rate limit reached for gpt-4o in organization xxxxxxxxxx on tokens per min (TPM): Limit 30000, Used 25317, Requested 8450. Please try again in 7.534s. Visit https://platform.openai.com/account/rate-limits to learn more.) (type . tokens) (param) (code . rate_limit_exceeded)))
It was a long prompt (27KB text).
For now, I added a 60s sleep before all calls to openai-chat
and I never get this error anymore. But this is too conservative. Ideally, openai-chat
should have an option to automatically try after waiting for the time mentioned in the error message (7.534s in this case)
Do any one have a variant of openai-chat that would automatically handle rate limit: retry automatically after the time openai suggests in the error?