Support long-lasting reserve / release

[kskalski]

I couldn't find the API for the following use case:

• rate limit bursts of requests based on time period
• also support long lasting requests that continuously "take up" certain amount of quota until they are finished and release happens on the limiter

AFAIU the current API assumes that cost tracked by limiter is "instantaneous" and only checks that the sum of cost incurred in period of time is capped. Could it be extended to arbitrarily long time intervals, where basically cost is assumed to be present at current time point until removed manually through API.

let n = NonZeroU32::new(10).unwrap();
let limiter = DefaultDirectRateLimiter::direct(Quota::per_minute(NonZeroU32::new(100).unwrap()));
limiter.until_n_reserve(n).await;
// start using n resource units, rate limiter's quota is decreased until release happens
run(n).await;
limiter.release(n);
// from this point on limiter counts time when minute bucket fully is available

Possibly the API could return an explicit reservation object, that would release automatically upon drop.

[antifuchs]

The data model underlying governor doesn't support this (and can't support it, I believe): What you're asking for would be easy with a token bucket (where you suspend the generation of tokens and add them on .release), but the GCRA doesn't work like that: It posits that there is a "theoretical arrival time" that is pre-computed for the next thing; and when the next thing comes in after that time, it's automatically accepted.

That's kinda the tradeoff you make with GCRA: it's performant and extremely cheap in terms of memory; but you do lose that flexibility to reserve capacity and then return it. I would prefer to keep governor's main data types' scope restricted to the current kind of use case, where consumers check if their predicted cost (and it can only be a prediction) can be covered, and then don't consume resources if it can not be.

Finding a solution for your use case might still be possible though: Have you checked the allow_burst setting for Quotas? It (in addition to check_n or until_n) might allow formulating a pool of api run time that workers can take a "loan" out.

[kskalski]

Thanks for advice, I reassessed my use-case and I don't think I will actually need this feature. I'm having some trouble with rate limiter admitting too many requests at the initial stage (for per minute quota), but that's a completely different issue, so I will close this one now.