search

edubruell / tidyllm

An tidy interface to large language model APIs for R
https://edubruell.github.io/tidyllm/
Other
36 stars 2 forks source link

Fix a bug in the ratelimit behaviour #25

Closed edubruell closed 1 month ago

edubruell commented 1 month ago

First of all here is a message that is wrong: Waiting till the token rate limit is reset: round(56.8641650676727,2) seconds because I have the round()-statement likely outside glue or paste0().

Secondly, i should set .min_token_reset to a higher value than 0 in the API functions. Usually that should be slightly larger than the token-reset of the average API-call. Perhaps setting it to 0, and saving the token-use metadata of the last API call or the change in tokens for rate-limiting would be a good approach to fix this. Till a fix is out, i could upgrade the documentation, to make clearer why this is needed.

edubruell commented 1 month ago

Glue statement had a problem. Still should add a sentence or two how not to run into rate limits.