Should there be an initial burst? Or an option to control that?
On the one hand, if you assume that we're starting from a blank slate, where everything's been idle infinitely far in the past, then it makes sense to start out the bucket with the full max_burst tokens in it.
On the other hand, if we assume that we have no idea about the past, and maybe some other process or call or something might have used up our quota just before this call started, then the conservative thing to do is to start the bucket with 0 tokens. This is the only way to guarantee that we don't exceed the max_per_second limit over the course of a call.
Right now what we do is start the bucket with 1 token, which is hard to justify theoretically! But it means that we immediately kick off the first task, and then makes the next task wait (1/max_per_second) seconds to start, which is possibly what people would naively expect...
Should there be an initial burst? Or an option to control that?
On the one hand, if you assume that we're starting from a blank slate, where everything's been idle infinitely far in the past, then it makes sense to start out the bucket with the full
max_bursttokens in it.On the other hand, if we assume that we have no idea about the past, and maybe some other process or call or something might have used up our quota just before this call started, then the conservative thing to do is to start the bucket with 0 tokens. This is the only way to guarantee that we don't exceed the
max_per_secondlimit over the course of a call.Right now what we do is start the bucket with 1 token, which is hard to justify theoretically! But it means that we immediately kick off the first task, and then makes the next task wait (1/max_per_second) seconds to start, which is possibly what people would naively expect...