Implement Rate Limiting on Requests

[ardatosun]

To enhance the stability and performance of our load balancer, we need to introduce a rate limiting feature that restricts the number of requests sent to each backend service to a specified limit per second. This implementation will provide a default global limit, with the ability to override this limit for individual backends.

Requirements:

• Global Rate Limit: Define a default global rate limit for requests per second across all backends.

• Override for Individual Backends: Allow each backend to specify its own rate limit, overriding the global setting.

• Configuration Options:

  • A global configuration option (e.g., global_requests_per_second) for the default rate limit.
  • Backend-specific configuration (e.g., backend_requests_per_second) to set individual limits.

• Handle Exceeding Limits: Ensure that appropriate responses are returned when the defined limit is exceeded (e.g., HTTP 429 Too Many Requests).

[ohyesgocool]

Hi @ardatosun I can pick this up if that is okay ?

[ardatosun]

You are on fire @ohyesgocool, sure 😄