add ratelimiter & load shedding

[komuw]

see;

1. https://pkg.go.dev/golang.org/x/time/rate
2. https://github.com/uber-go/ratelimit
3. https://github.com/juju/ratelimit
4. https://github.com/gofiber/fiber/tree/master/middleware/limiter
5. https://github.com/labstack/echo/blob/master/middleware/rate_limiter.go
6. https://github.com/asecurityteam/loadshed

[komuw]

https://fasterthanli.me/articles/i-won-free-load-testing

[komuw]

I think we should have two(possibly more) ratelimiters;

• by IP address. Maybe like 1req/sec.
• by IP range. https://stackoverflow.com/a/40072896/2768067. Maybe like 1000req/sec.

I'm not sure whether we want to ratelimit by IP range. What if most requests are coming from the cloudflare IP range? It is not co-ordinated attack, it is just unrelated customers of cloudflare.

[komuw]

I have tuned that to do rate limiting, circuit breaking (stop sending requests to an upstream when 
it returns too many errors), idle connection termination, and to shed load when a certain amount 
of memory is in use, so without any additional configuration for a new service behind the proxy it's 
somewhat difficult to get the proxy and other services to not respond at all.
I use a custom rate limit service to aggregate rate limits across a /24.

from https://news.ycombinator.com/item?id=31231726

[komuw]

note:

• if you have an IP address like 105.162.13.216 then it's /24 range is 105.162.13.0 to 105.162.13.255 and its CIDR is 105.162.13.0/24

[komuw]

related: https://github.com/komuw/goweb/issues/8

[komuw]

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After

[komuw]

checkout: https://aws.amazon.com/builders-library/using-load-shedding-to-avoid-overload/

[komuw]

I'm not sure whether we want to ratelimit by IP range. What if most requests are coming from the cloudflare IP range? It is not co-ordinated attack, it is just unrelated customers of cloudflare.

In [1], it is argued that you could have multiple rate-limiters; one that limits per-ip, another that limits per /64, another per /48 etc.

See also[2] on how golang/pkgsite/ does ratelimiting.

1. https://adam-p.ca/blog/2022/02/ipv6-rate-limiting/
2. https://github.com/golang/pkgsite/blob/8b9886cdffa22a4f43206ac784fa1646fd0b6da6/internal/middleware/quota.go#L50-L70

[komuw]

All about retry/retries:

• https://medium.com/yandex/good-retry-bad-retry-an-incident-story-648072d3cee6
• https://brooker.co.za/blog/2022/02/28/retries.html
• https://brooker.co.za/blog/2022/08/11/backoff.html
• https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/