Rate Limiting

[dfarr]

User provided functions can be rate limited. Rate limiters can be configured on the resonate instance and assigned an id, if a user provided function specifies this id in the options then the function will be rate limited according to that rate limit policy (along with any other function invocations against the same rate limiter).

[vaibhawvipul]

	RocksDB	Wdt	Apache Spark	Apache Kafka	Google Guava
Type	Leaky Bucket	Token Bucket	Token Bucket	Leaky Bucket	Token Bucket
Enforce Avg Rate	Yes	Yes	Yes	Yes	Yes
Short Bursts		Yes	Yes		Yes
Warm-up			Yes		Yes
Alter Rate	Yes	Yes	Yes	Yes	Yes
Priority	Yes

Choosing the Right Algorithm:

Consider Rate Enforcement: If strict enforcement of the average rate is necessary, algorithms like Token Bucket are suitable. Evaluate Burst Tolerance: If short bursts exceeding the average rate are permissible, algorithms like Token Bucket are preferable over Leaky Bucket. Assess Runtime Adjustments: If dynamic rate adjustments are required, algorithms supporting alteration at runtime, such as Token Bucket, are advantageous. Factor in Warm-up Requirements: If warm-up functionality after an idle period is needed, algorithms like Token Bucket with warm-up capabilities are appropriate. Prioritize Stability vs. Flexibility: Choose an algorithm based on whether stability (Leaky Bucket) or flexibility (Token Bucket) is more critical for your specific use case.

Recommending Token Bucket.

cc @dtornow @guergabo @dfarr

[dfarr]

Nice! Thanks for the research. Whichever we choose we should implement it against an interface and allow users to pass in their own rate implementation if needed.