Closed ewdurbin closed 1 month ago
It appears rate limiting is working...but its the load-balancers that are rate limited. Logs indicate that lb-a and lb-b are being limited via their internal IPs
2024/07/25 19:55:07 [error] 2527603#2527603: *40848 limiting requests, excess: 5.110 by zone "limit-cpython", client: 10.132.141.7, server: bugs.python.org, request: "GET /issue16136 HTTP/1.1", host: "bugs.python.org"
2024/07/25 19:55:08 [error] 2527603#2527603: *40855 limiting requests, excess: 5.650 by zone "limit-cpython", client: 10.132.68.69, server: bugs.python.org, request: "GET /issue46282 HTTP/1.1", host: "bugs.python.org"
Ostensibly we are rate limiting access to our roundup instances in the bugs role.
But it appears that our limits are either not working, or are mistuned.
We should debug why rate limits aren't keeping us from seeing over-load outages like the one on July 7.