search

InAnYan / jabref

Graphical Java application for managing BibTeX and biblatex (.bib) databases
https://devdocs.jabref.org
MIT License
0 stars 0 forks source link

Handle rate limit for <3 seconds #107

Open koppor opened 3 months ago

koppor commented 3 months ago

429 - message: Rate limit reached for gpt-4 in organization org-3M84iZQTliRuKpVVeuB53iHJ on tokens per min (TPM): Limit 10000, Used 8141, Requested 2071. Please try again in 1.272s. Visit https://platform.openai.com/account/rate-limits to learn more., type: tokens, param: null, code: rate_limit_exceeded

I think, JabRef can for itself retry if the rate limit is reached and below 3 seconds for a retry.

There nevertheless should be a log message.

InAnYan commented 3 months ago

This unfortunately links to: https://github.com/InAnYan/jabref/issues/23.

So, there is no way to do this. Will add more comments to the referenced issue

koppor commented 3 months ago

So, there is no way to do this. Will add more comments to the referenced issue

I don't understand.

I put my assumptions numbered. Please say, which is wrong.

  1. JabRef receives the answer of OpenAI
  2. JabRef can read the answer as text
  3. JabRef can look for the pattern "Please try again in x.ys" in the answer text
  4. JabRef can convert x to an Integer i.
  5. JabRef can check if i is greater less than 3.
  6. JabRef can resend the request instead of outputting the errror.
InAnYan commented 3 months ago

Oh, of course we can do this.

It's just it's hacky. What if some service has other pattern?

But okay, I'll try to implement this

koppor commented 3 months ago

It's just it's hacky. What if some service has other pattern?

Then we will see it and add support for it on a later point in time. - This is the consequence if there are non-standardized messages. -- Timeout exceptions should be made by langchain4j. However, they do the opposite: RuntimeExceptions 🙈