search

emacs-openai / openai

Elisp library for the OpenAI API
GNU General Public License v3.0
103 stars 17 forks source link

automatically handle rate limit #33

Open aa755 opened 4 months ago

aa755 commented 4 months ago

Do any one have a variant of openai-chat that would automatically handle rate limit: retry automatically after the time openai suggests in the error?

jcs090218 commented 4 months ago

Not sure what to handle. The rate limit means your account is running out of quota. 🤔 Retry wouldn't help in these circumstances.

aa755 commented 4 months ago

Here is an example response I was getting from the openai-chat function in this repo:

((error (message . Rate limit reached for gpt-4o in organization xxxxxxxxxx on tokens per min (TPM): Limit 30000, Used 25317, Requested 8450. Please try again in 7.534s. Visit https://platform.openai.com/account/rate-limits to learn more.) (type . tokens) (param) (code . rate_limit_exceeded)))

It was a long prompt (27KB text). For now, I added a 60s sleep before all calls to openai-chat and I never get this error anymore. But this is too conservative. Ideally, openai-chat should have an option to automatically try after waiting for the time mentioned in the error message (7.534s in this case)