Closed eguzki closed 5 months ago
@alexsnaps interesting to note that ping is about 10ms (done with ping
just now) and the average latency is double ~21ms.
This is maybe because we are doing two RTT (one for read counters, other for writting)??
Noting also that this setup is unconventional, we then might want to run the same test when we have setup using Transit Gateways
This is maybe because we are doing two RTT (one for read counters, other for writing)??
Yes... there are multiple roundtrips to Redis... added to that, there is the work redis has to do, encoding/decoding the requests, serializing/deserializing the payloads... I get it that the latter are CPU bound and not necessarily affected by the network... tho we are still not as lightweight as a ping
.
We do tho:
Adding this new test for the record: 3 limts
---
- namespace: test_namespace
max_value: 100000
seconds: 5
conditions:
variables:
- namespace: test_namespace
max_value: 100000
seconds: 4
conditions:
variables:
- namespace: test_namespace
max_value: 100000
seconds: 3
conditions:
variables:
When there is 1 limit, limitador needs 1 RTT to get all the counter values and 1 RTT to increment. Total 2*RTT.
When there are 3 limit, limitador needs 1 RTT to get all the counter values and 3 RTT to increment them. Total 4*RTT.
Result
$ docker run --rm -v $PWD/limits.yaml:/bench/limits.yaml quay.io/kuadrant/limitador-driver:v0.0.1-alpha-draft storage=redis storage_url=redis://perftest-limitador.0bg45u.ng.0001.use1.cache.amazonaws.com:6379 time=1m
Parsing arguments
Executing limitador: $ limitador-server /bench/limits.yaml redis redis://perftest-limitador.0bg45u.ng.0001.use1.cache.amazonaws.com:6379
Limitador Server v1.4.0-dev (b47cabab) [] release
Executing limitador-driver: $ limitador-driver rpc://127.0.0.1:8081 1m
Hitting Limitador at rpc://127.0.0.1:8081 for a total duration of 1 minutes
Error: tonic::transport::Error(Transport, hyper::Error(Connect, ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })))
[ec2-user@ip-172-31-30-13 ~]$ docker run --rm -v $PWD/limits.yaml:/bench/limits.yaml quay.io/kuadrant/limitador-driver:v0.0.1-alpha-draft storage=redis storage_url=redis://ec2-3-83-231-224.compute-1.amazonaws.com:9876 time=1m
Parsing arguments
Executing limitador: $ limitador-server /bench/limits.yaml redis redis://ec2-3-83-231-224.compute-1.amazonaws.com:9876
Limitador Server v1.4.0-dev (b47cabab) [] release
Executing limitador-driver: $ limitador-driver rpc://127.0.0.1:8081 1m
Hitting Limitador at rpc://127.0.0.1:8081 for a total duration of 1 minutes
Current: hits: 23, mean: 43.915ms, p50: 43.263ms, p90: 43.519ms, p99: 63.231ms, p999: 63.231ms, max: 63.231ms
Current: hits: 24, mean: 43.104ms, p50: 43.263ms, p90: 43.519ms, p99: 43.775ms, p999: 43.775ms, max: 43.775ms
Current: hits: 24, mean: 42.603ms, p50: 42.751ms, p90: 43.263ms, p99: 43.263ms, p999: 43.263ms, max: 43.263ms
Current: hits: 24, mean: 42.507ms, p50: 42.495ms, p90: 42.751ms, p99: 43.007ms, p999: 43.007ms, max: 43.007ms
Current: hits: 24, mean: 42.699ms, p50: 42.751ms, p90: 43.263ms, p99: 43.263ms, p999: 43.263ms, max: 43.263ms
Current: hits: 24, mean: 42.677ms, p50: 42.751ms, p90: 43.263ms, p99: 43.263ms, p999: 43.263ms, max: 43.263ms
Current: hits: 24, mean: 42.421ms, p50: 42.495ms, p90: 42.751ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 43.328ms, p50: 43.007ms, p90: 43.519ms, p99: 51.455ms, p999: 51.455ms, max: 51.455ms
Current: hits: 24, mean: 42.976ms, p50: 43.007ms, p90: 43.263ms, p99: 43.775ms, p999: 43.775ms, max: 43.775ms
Current: hits: 24, mean: 42.752ms, p50: 42.751ms, p90: 43.007ms, p99: 43.263ms, p999: 43.263ms, max: 43.263ms
Current: hits: 24, mean: 42.571ms, p50: 42.751ms, p90: 43.007ms, p99: 43.007ms, p999: 43.007ms, max: 43.007ms
Current: hits: 24, mean: 42.560ms, p50: 42.751ms, p90: 43.007ms, p99: 43.263ms, p999: 43.263ms, max: 43.263ms
Current: hits: 24, mean: 42.837ms, p50: 43.007ms, p90: 43.263ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 43.019ms, p50: 43.263ms, p90: 43.519ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 42.891ms, p50: 43.007ms, p90: 43.263ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 42.443ms, p50: 42.495ms, p90: 42.751ms, p99: 43.007ms, p999: 43.007ms, max: 43.007ms
Current: hits: 24, mean: 42.837ms, p50: 43.007ms, p90: 43.263ms, p99: 43.263ms, p999: 43.263ms, max: 43.263ms
Current: hits: 24, mean: 42.859ms, p50: 43.007ms, p90: 43.263ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 42.933ms, p50: 43.007ms, p90: 43.263ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 42.752ms, p50: 42.751ms, p90: 43.263ms, p99: 43.263ms, p999: 43.263ms, max: 43.263ms
Current: hits: 24, mean: 42.912ms, p50: 43.007ms, p90: 43.263ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 42.997ms, p50: 43.263ms, p90: 43.263ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 43.040ms, p50: 43.263ms, p90: 43.519ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 42.581ms, p50: 42.751ms, p90: 43.007ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 42.517ms, p50: 42.495ms, p90: 43.007ms, p99: 43.007ms, p999: 43.007ms, max: 43.007ms
Current: hits: 24, mean: 42.443ms, p50: 42.495ms, p90: 43.007ms, p99: 43.007ms, p999: 43.007ms, max: 43.007ms
Current: hits: 24, mean: 42.443ms, p50: 42.495ms, p90: 42.751ms, p99: 43.007ms, p999: 43.007ms, max: 43.007ms
Current: hits: 24, mean: 42.859ms, p50: 43.007ms, p90: 43.519ms, p99: 43.775ms, p999: 43.775ms, max: 43.775ms
Current: hits: 24, mean: 42.965ms, p50: 43.007ms, p90: 43.263ms, p99: 43.775ms, p999: 43.775ms, max: 43.775ms
Current: hits: 24, mean: 42.891ms, p50: 43.007ms, p90: 43.263ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 43.008ms, p50: 43.007ms, p90: 43.519ms, p99: 43.775ms, p999: 43.775ms, max: 43.775ms
Current: hits: 24, mean: 42.880ms, p50: 43.007ms, p90: 43.263ms, p99: 43.263ms, p999: 43.263ms, max: 43.263ms
Current: hits: 24, mean: 42.891ms, p50: 43.007ms, p90: 43.519ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 42.741ms, p50: 42.751ms, p90: 43.263ms, p99: 43.263ms, p999: 43.263ms, max: 43.263ms
Current: hits: 24, mean: 42.869ms, p50: 43.007ms, p90: 43.519ms, p99: 43.775ms, p999: 43.775ms, max: 43.775ms
Current: hits: 24, mean: 42.571ms, p50: 42.751ms, p90: 43.007ms, p99: 43.007ms, p999: 43.007ms, max: 43.007ms
Current: hits: 24, mean: 42.987ms, p50: 43.007ms, p90: 43.263ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 42.848ms, p50: 43.007ms, p90: 43.519ms, p99: 43.775ms, p999: 43.775ms, max: 43.775ms
Current: hits: 24, mean: 42.965ms, p50: 43.007ms, p90: 43.519ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 43.083ms, p50: 43.263ms, p90: 43.519ms, p99: 43.775ms, p999: 43.775ms, max: 43.775ms
Current: hits: 24, mean: 42.784ms, p50: 42.751ms, p90: 43.519ms, p99: 43.775ms, p999: 43.775ms, max: 43.775ms
Current: hits: 24, mean: 43.093ms, p50: 43.263ms, p90: 43.519ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 42.859ms, p50: 43.007ms, p90: 43.263ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 42.752ms, p50: 42.751ms, p90: 43.263ms, p99: 43.263ms, p999: 43.263ms, max: 43.263ms
Current: hits: 24, mean: 42.816ms, p50: 43.007ms, p90: 43.263ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 42.987ms, p50: 43.263ms, p90: 43.263ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 43.019ms, p50: 43.007ms, p90: 43.519ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 42.987ms, p50: 43.263ms, p90: 43.519ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 43.115ms, p50: 43.263ms, p90: 43.519ms, p99: 43.775ms, p999: 43.775ms, max: 43.775ms
Current: hits: 24, mean: 42.720ms, p50: 42.751ms, p90: 43.007ms, p99: 43.263ms, p999: 43.263ms, max: 43.263ms
Current: hits: 24, mean: 42.603ms, p50: 42.751ms, p90: 43.007ms, p99: 43.007ms, p999: 43.007ms, max: 43.007ms
Current: hits: 24, mean: 42.741ms, p50: 43.007ms, p90: 43.263ms, p99: 43.263ms, p999: 43.263ms, max: 43.263ms
Current: hits: 24, mean: 42.912ms, p50: 43.007ms, p90: 43.519ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 42.581ms, p50: 42.751ms, p90: 43.007ms, p99: 43.263ms, p999: 43.263ms, max: 43.263ms
Current: hits: 24, mean: 42.517ms, p50: 42.495ms, p90: 43.007ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 42.891ms, p50: 43.007ms, p90: 43.263ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 43.019ms, p50: 43.263ms, p90: 43.263ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
Current: hits: 24, mean: 43.019ms, p50: 43.007ms, p90: 43.519ms, p99: 43.519ms, p999: 43.519ms, max: 43.519ms
============== Done! =================
Ok: hits: 1401, mean: 42.838ms, p50: 43.007ms, p90: 43.263ms, p99: 43.775ms, p999: 51.455ms, max: 63.231ms
OverLimit: hits: 0, mean: 0.000ms, p50: 0.000ms, p90: 0.000ms, p99: 0.000ms, p999: 0.000ms, max: 0.000ms
Unknown: hits: 0, mean: 0.000ms, p50: 0.000ms, p90: 0.000ms, p99: 0.000ms, p999: 0.000ms, max: 0.000ms
Overall: hits: 1401, mean: 42.838ms, p50: 43.007ms, p90: 43.263ms, p99: 43.775ms, p999: 51.455ms, max: 63.231ms
4.38ms | | 0.0th %-ile
8.75ms | | 0.0th %-ile
13.13ms | | 0.0th %-ile
17.51ms | | 0.0th %-ile
21.89ms | | 0.0th %-ile
26.26ms | | 0.0th %-ile
30.64ms | | 0.0th %-ile
35.02ms | | 0.0th %-ile
39.39ms | | 0.0th %-ile
43.77ms | ******************************************************************************** | 99.9th %-ile
48.15ms | | 99.9th %-ile
52.52ms | * | 99.9th %-ile
The latency is x2 as expected
Adding this new test for the record: redis cache with defaults
Uses Redis to store counters, with an in-memory cache
Usage: limitador-server <LIMITS_FILE> redis_cached [OPTIONS] <URL>
Arguments:
<URL> Redis URL to use
Options:
--ttl <TTL> TTL for cached counters in milliseconds [default: 5000]
--ratio <ratio> Ratio to apply to the TTL from Redis on cached counters [default: 10000]
--flush-period <flush> Flushing period for counters in milliseconds [default: 1000]
--max-cached <max> Maximum amount of counters cached [default: 10000]
-h, --help Print hel
Executing limitador: $ limitador-server /bench/limits.yaml redis_cached redis://ec2-34-224-88-175.compute-1.amazonaws.com:9876
Limitador Server v1.4.0-dev (b47cabab) [] release
Executing limitador-driver: $ limitador-driver rpc://127.0.0.1:8081 1m
Hitting Limitador at rpc://127.0.0.1:8081 for a total duration of 1 minutes
Current: hits: 88, mean: 11.373ms, p50: 10.815ms, p90: 10.943ms, p99: 41.215ms, p999: 41.215ms, max: 41.215ms
Current: hits: 92, mean: 10.878ms, p50: 10.815ms, p90: 10.943ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
Current: hits: 93, mean: 10.857ms, p50: 10.815ms, p90: 10.943ms, p99: 20.863ms, p999: 20.863ms, max: 20.863ms
Current: hits: 93, mean: 10.850ms, p50: 10.815ms, p90: 10.943ms, p99: 20.991ms, p999: 20.991ms, max: 20.991ms
Current: hits: 92, mean: 10.878ms, p50: 10.751ms, p90: 11.007ms, p99: 21.247ms, p999: 21.247ms, max: 21.247ms
Current: hits: 92, mean: 10.891ms, p50: 10.815ms, p90: 10.943ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
Current: hits: 92, mean: 10.956ms, p50: 10.815ms, p90: 10.943ms, p99: 20.991ms, p999: 20.991ms, max: 20.991ms
Current: hits: 92, mean: 10.881ms, p50: 10.815ms, p90: 10.879ms, p99: 21.247ms, p999: 21.247ms, max: 21.247ms
Current: hits: 92, mean: 10.878ms, p50: 10.815ms, p90: 10.943ms, p99: 20.863ms, p999: 20.863ms, max: 20.863ms
Current: hits: 93, mean: 10.844ms, p50: 10.751ms, p90: 10.943ms, p99: 20.991ms, p999: 20.991ms, max: 20.991ms
Current: hits: 94, mean: 10.698ms, p50: 10.687ms, p90: 10.879ms, p99: 11.007ms, p999: 11.007ms, max: 11.007ms
Current: hits: 92, mean: 10.870ms, p50: 10.751ms, p90: 10.943ms, p99: 21.375ms, p999: 21.375ms, max: 21.375ms
Current: hits: 93, mean: 10.846ms, p50: 10.751ms, p90: 10.943ms, p99: 21.247ms, p999: 21.247ms, max: 21.247ms
Current: hits: 93, mean: 10.772ms, p50: 10.687ms, p90: 10.815ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
Current: hits: 93, mean: 10.809ms, p50: 10.687ms, p90: 10.815ms, p99: 24.319ms, p999: 24.319ms, max: 24.319ms
Current: hits: 93, mean: 10.803ms, p50: 10.751ms, p90: 10.879ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
Current: hits: 92, mean: 10.899ms, p50: 10.751ms, p90: 10.943ms, p99: 20.863ms, p999: 20.863ms, max: 20.863ms
Current: hits: 93, mean: 10.854ms, p50: 10.751ms, p90: 10.943ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
Current: hits: 93, mean: 10.862ms, p50: 10.751ms, p90: 10.943ms, p99: 21.247ms, p999: 21.247ms, max: 21.247ms
Current: hits: 93, mean: 10.822ms, p50: 10.751ms, p90: 10.879ms, p99: 20.991ms, p999: 20.991ms, max: 20.991ms
Current: hits: 92, mean: 10.877ms, p50: 10.815ms, p90: 10.943ms, p99: 20.863ms, p999: 20.863ms, max: 20.863ms
Current: hits: 92, mean: 10.912ms, p50: 10.815ms, p90: 10.943ms, p99: 21.375ms, p999: 21.375ms, max: 21.375ms
Current: hits: 93, mean: 10.862ms, p50: 10.815ms, p90: 10.943ms, p99: 20.735ms, p999: 20.735ms, max: 20.735ms
Current: hits: 92, mean: 10.880ms, p50: 10.751ms, p90: 10.943ms, p99: 21.247ms, p999: 21.247ms, max: 21.247ms
Current: hits: 93, mean: 10.835ms, p50: 10.751ms, p90: 10.943ms, p99: 21.375ms, p999: 21.375ms, max: 21.375ms
Current: hits: 92, mean: 10.874ms, p50: 10.751ms, p90: 10.943ms, p99: 21.247ms, p999: 21.247ms, max: 21.247ms
Current: hits: 93, mean: 10.938ms, p50: 10.751ms, p90: 10.879ms, p99: 20.991ms, p999: 20.991ms, max: 20.991ms
Current: hits: 93, mean: 10.836ms, p50: 10.751ms, p90: 10.879ms, p99: 20.991ms, p999: 20.991ms, max: 20.991ms
Current: hits: 92, mean: 10.880ms, p50: 10.815ms, p90: 11.007ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
Current: hits: 92, mean: 10.917ms, p50: 10.815ms, p90: 11.007ms, p99: 21.247ms, p999: 21.247ms, max: 21.247ms
Current: hits: 92, mean: 10.888ms, p50: 10.815ms, p90: 10.943ms, p99: 21.247ms, p999: 21.247ms, max: 21.247ms
Current: hits: 92, mean: 10.893ms, p50: 10.815ms, p90: 11.007ms, p99: 20.991ms, p999: 20.991ms, max: 20.991ms
Current: hits: 92, mean: 10.875ms, p50: 10.815ms, p90: 10.943ms, p99: 20.991ms, p999: 20.991ms, max: 20.991ms
Current: hits: 92, mean: 10.880ms, p50: 10.815ms, p90: 10.943ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
Current: hits: 93, mean: 10.856ms, p50: 10.751ms, p90: 10.943ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
Current: hits: 92, mean: 10.923ms, p50: 10.815ms, p90: 11.007ms, p99: 21.375ms, p999: 21.375ms, max: 21.375ms
Current: hits: 93, mean: 10.795ms, p50: 10.687ms, p90: 10.815ms, p99: 20.863ms, p999: 20.863ms, max: 20.863ms
Current: hits: 92, mean: 10.875ms, p50: 10.815ms, p90: 10.943ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
Current: hits: 92, mean: 10.901ms, p50: 10.815ms, p90: 11.007ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
Current: hits: 93, mean: 10.815ms, p50: 10.751ms, p90: 10.943ms, p99: 20.991ms, p999: 20.991ms, max: 20.991ms
Current: hits: 93, mean: 10.795ms, p50: 10.751ms, p90: 10.879ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
Current: hits: 93, mean: 10.813ms, p50: 10.751ms, p90: 10.815ms, p99: 20.991ms, p999: 20.991ms, max: 20.991ms
Current: hits: 93, mean: 10.805ms, p50: 10.751ms, p90: 10.879ms, p99: 20.991ms, p999: 20.991ms, max: 20.991ms
Current: hits: 93, mean: 10.825ms, p50: 10.751ms, p90: 10.879ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
Current: hits: 93, mean: 10.833ms, p50: 10.751ms, p90: 10.879ms, p99: 21.247ms, p999: 21.247ms, max: 21.247ms
Current: hits: 93, mean: 10.834ms, p50: 10.751ms, p90: 10.879ms, p99: 21.631ms, p999: 21.631ms, max: 21.631ms
Current: hits: 92, mean: 10.907ms, p50: 10.815ms, p90: 10.943ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
Current: hits: 94, mean: 10.719ms, p50: 10.751ms, p90: 10.879ms, p99: 11.071ms, p999: 11.071ms, max: 11.071ms
Current: hits: 92, mean: 10.895ms, p50: 10.815ms, p90: 10.943ms, p99: 21.247ms, p999: 21.247ms, max: 21.247ms
Current: hits: 92, mean: 10.886ms, p50: 10.815ms, p90: 10.943ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
Current: hits: 92, mean: 10.921ms, p50: 10.815ms, p90: 11.007ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
Current: hits: 93, mean: 10.813ms, p50: 10.751ms, p90: 10.879ms, p99: 20.991ms, p999: 20.991ms, max: 20.991ms
Current: hits: 93, mean: 10.839ms, p50: 10.751ms, p90: 10.879ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
Current: hits: 93, mean: 10.825ms, p50: 10.751ms, p90: 10.879ms, p99: 20.991ms, p999: 20.991ms, max: 20.991ms
Current: hits: 92, mean: 10.899ms, p50: 10.815ms, p90: 10.943ms, p99: 20.991ms, p999: 20.991ms, max: 20.991ms
Current: hits: 92, mean: 10.903ms, p50: 10.815ms, p90: 11.007ms, p99: 20.991ms, p999: 20.991ms, max: 20.991ms
Current: hits: 93, mean: 10.843ms, p50: 10.751ms, p90: 10.943ms, p99: 21.247ms, p999: 21.247ms, max: 21.247ms
Current: hits: 92, mean: 10.889ms, p50: 10.815ms, p90: 10.943ms, p99: 21.247ms, p999: 21.247ms, max: 21.247ms
Current: hits: 92, mean: 10.935ms, p50: 10.879ms, p90: 11.071ms, p99: 21.119ms, p999: 21.119ms, max: 21.119ms
============== Done! =================
Ok: hits: 5522, mean: 10.867ms, p50: 10.751ms, p90: 10.943ms, p99: 20.863ms, p999: 21.375ms, max: 41.215ms
OverLimit: hits: 0, mean: 0.000ms, p50: 0.000ms, p90: 0.000ms, p99: 0.000ms, p999: 0.000ms, max: 0.000ms
Unknown: hits: 0, mean: 0.000ms, p50: 0.000ms, p90: 0.000ms, p99: 0.000ms, p999: 0.000ms, max: 0.000ms
Overall: hits: 5522, mean: 10.867ms, p50: 10.751ms, p90: 10.943ms, p99: 20.863ms, p999: 21.375ms, max: 41.215ms
2.09ms | | 0.0th %-ile
4.17ms | | 0.0th %-ile
6.26ms | | 0.0th %-ile
8.34ms | | 0.0th %-ile
10.43ms | * | 0.5th %-ile
12.52ms | ******************************************************************************* | 98.9th %-ile
14.60ms | * | 98.9th %-ile
16.69ms | * | 98.9th %-ile
18.77ms | * | 98.9th %-ile
20.86ms | * | 99.0th %-ile
22.95ms | * | 99.9th %-ile
Latency is 1 RTT (10ms ping)
Adding this new test for the record: redis cache with --ratio 1
doc
/bench/limitador-server /bench/limits.yaml redis_cached --ratio 1 redis://ec2-34-224-88-175.compute-1.amazonaws.com:9876
root@cd25c8659ec6:/bench# /bench/limitador-driver rpc://127.0.0.1:8081 1m
Hitting Limitador at rpc://127.0.0.1:8081 for a total duration of 1 minutes
Current: hits: 12765, mean: 0.078ms, p50: 0.075ms, p90: 0.085ms, p99: 0.106ms, p999: 0.271ms, max: 11.327ms
Current: hits: 12953, mean: 0.076ms, p50: 0.075ms, p90: 0.084ms, p99: 0.106ms, p999: 0.267ms, max: 11.071ms
Current: hits: 12879, mean: 0.077ms, p50: 0.075ms, p90: 0.085ms, p99: 0.107ms, p999: 0.259ms, max: 10.943ms
Current: hits: 12843, mean: 0.077ms, p50: 0.075ms, p90: 0.085ms, p99: 0.109ms, p999: 0.257ms, max: 11.583ms
Current: hits: 12926, mean: 0.077ms, p50: 0.075ms, p90: 0.085ms, p99: 0.107ms, p999: 0.252ms, max: 10.879ms
Current: hits: 12418, mean: 0.080ms, p50: 0.075ms, p90: 0.085ms, p99: 0.109ms, p999: 0.299ms, max: 20.991ms
Current: hits: 12829, mean: 0.077ms, p50: 0.075ms, p90: 0.085ms, p99: 0.108ms, p999: 0.243ms, max: 10.943ms
Current: hits: 12860, mean: 0.077ms, p50: 0.075ms, p90: 0.085ms, p99: 0.111ms, p999: 0.251ms, max: 10.751ms
Current: hits: 12753, mean: 0.078ms, p50: 0.076ms, p90: 0.086ms, p99: 0.109ms, p999: 0.254ms, max: 10.623ms
Current: hits: 12688, mean: 0.078ms, p50: 0.076ms, p90: 0.086ms, p99: 0.111ms, p999: 0.269ms, max: 10.559ms
Current: hits: 12558, mean: 0.079ms, p50: 0.076ms, p90: 0.086ms, p99: 0.111ms, p999: 0.289ms, max: 11.007ms
Current: hits: 12720, mean: 0.078ms, p50: 0.076ms, p90: 0.086ms, p99: 0.110ms, p999: 0.254ms, max: 10.751ms
Current: hits: 12698, mean: 0.078ms, p50: 0.076ms, p90: 0.086ms, p99: 0.111ms, p999: 0.303ms, max: 10.943ms
Current: hits: 12763, mean: 0.078ms, p50: 0.075ms, p90: 0.086ms, p99: 0.114ms, p999: 0.265ms, max: 10.815ms
Current: hits: 12762, mean: 0.078ms, p50: 0.075ms, p90: 0.086ms, p99: 0.110ms, p999: 0.259ms, max: 10.751ms
Current: hits: 12569, mean: 0.079ms, p50: 0.076ms, p90: 0.086ms, p99: 0.106ms, p999: 0.285ms, max: 11.071ms
Current: hits: 12649, mean: 0.078ms, p50: 0.075ms, p90: 0.086ms, p99: 0.112ms, p999: 0.267ms, max: 21.119ms
Current: hits: 12749, mean: 0.078ms, p50: 0.076ms, p90: 0.086ms, p99: 0.109ms, p999: 0.269ms, max: 10.559ms
Current: hits: 12682, mean: 0.078ms, p50: 0.076ms, p90: 0.087ms, p99: 0.111ms, p999: 0.275ms, max: 10.879ms
Current: hits: 12720, mean: 0.078ms, p50: 0.076ms, p90: 0.086ms, p99: 0.110ms, p999: 0.263ms, max: 10.623ms
Current: hits: 12780, mean: 0.078ms, p50: 0.075ms, p90: 0.086ms, p99: 0.112ms, p999: 0.277ms, max: 11.775ms
Current: hits: 12502, mean: 0.079ms, p50: 0.075ms, p90: 0.086ms, p99: 0.108ms, p999: 0.267ms, max: 21.375ms
Current: hits: 12962, mean: 0.076ms, p50: 0.075ms, p90: 0.084ms, p99: 0.107ms, p999: 0.263ms, max: 11.135ms
Current: hits: 12987, mean: 0.076ms, p50: 0.075ms, p90: 0.084ms, p99: 0.107ms, p999: 0.255ms, max: 10.879ms
Current: hits: 13019, mean: 0.076ms, p50: 0.075ms, p90: 0.084ms, p99: 0.105ms, p999: 0.226ms, max: 11.007ms
Current: hits: 12952, mean: 0.076ms, p50: 0.075ms, p90: 0.084ms, p99: 0.103ms, p999: 0.245ms, max: 10.687ms
Current: hits: 12774, mean: 0.078ms, p50: 0.075ms, p90: 0.085ms, p99: 0.108ms, p999: 0.251ms, max: 21.375ms
Current: hits: 12944, mean: 0.077ms, p50: 0.075ms, p90: 0.084ms, p99: 0.108ms, p999: 0.240ms, max: 10.751ms
Current: hits: 12961, mean: 0.076ms, p50: 0.075ms, p90: 0.084ms, p99: 0.107ms, p999: 0.230ms, max: 10.943ms
Current: hits: 13086, mean: 0.076ms, p50: 0.074ms, p90: 0.083ms, p99: 0.104ms, p999: 0.233ms, max: 10.559ms
Current: hits: 13031, mean: 0.076ms, p50: 0.075ms, p90: 0.083ms, p99: 0.103ms, p999: 0.222ms, max: 10.879ms
Current: hits: 12692, mean: 0.078ms, p50: 0.075ms, p90: 0.084ms, p99: 0.109ms, p999: 0.259ms, max: 21.375ms
Current: hits: 12887, mean: 0.077ms, p50: 0.075ms, p90: 0.084ms, p99: 0.108ms, p999: 0.259ms, max: 10.943ms
Current: hits: 12901, mean: 0.077ms, p50: 0.075ms, p90: 0.084ms, p99: 0.105ms, p999: 0.244ms, max: 11.007ms
Current: hits: 12925, mean: 0.077ms, p50: 0.075ms, p90: 0.084ms, p99: 0.109ms, p999: 0.242ms, max: 10.751ms
Current: hits: 12796, mean: 0.077ms, p50: 0.076ms, p90: 0.085ms, p99: 0.109ms, p999: 0.244ms, max: 10.879ms
Current: hits: 12606, mean: 0.079ms, p50: 0.076ms, p90: 0.086ms, p99: 0.108ms, p999: 0.281ms, max: 10.879ms
Current: hits: 12555, mean: 0.079ms, p50: 0.075ms, p90: 0.087ms, p99: 0.129ms, p999: 0.287ms, max: 21.631ms
Current: hits: 12825, mean: 0.077ms, p50: 0.075ms, p90: 0.085ms, p99: 0.108ms, p999: 0.259ms, max: 10.751ms
Current: hits: 12678, mean: 0.078ms, p50: 0.076ms, p90: 0.086ms, p99: 0.113ms, p999: 0.281ms, max: 10.623ms
Current: hits: 12792, mean: 0.077ms, p50: 0.075ms, p90: 0.086ms, p99: 0.110ms, p999: 0.246ms, max: 10.815ms
Current: hits: 12785, mean: 0.077ms, p50: 0.075ms, p90: 0.086ms, p99: 0.112ms, p999: 0.265ms, max: 10.879ms
Current: hits: 12458, mean: 0.080ms, p50: 0.076ms, p90: 0.086ms, p99: 0.116ms, p999: 0.267ms, max: 21.375ms
Current: hits: 12694, mean: 0.078ms, p50: 0.076ms, p90: 0.086ms, p99: 0.114ms, p999: 0.279ms, max: 10.815ms
Current: hits: 12809, mean: 0.077ms, p50: 0.075ms, p90: 0.086ms, p99: 0.108ms, p999: 0.267ms, max: 10.623ms
Current: hits: 12758, mean: 0.078ms, p50: 0.076ms, p90: 0.086ms, p99: 0.109ms, p999: 0.267ms, max: 10.495ms
Current: hits: 12758, mean: 0.078ms, p50: 0.076ms, p90: 0.086ms, p99: 0.110ms, p999: 0.257ms, max: 10.879ms
Current: hits: 12461, mean: 0.080ms, p50: 0.076ms, p90: 0.086ms, p99: 0.112ms, p999: 0.285ms, max: 20.991ms
Current: hits: 12852, mean: 0.077ms, p50: 0.075ms, p90: 0.085ms, p99: 0.107ms, p999: 0.279ms, max: 10.751ms
Current: hits: 12811, mean: 0.077ms, p50: 0.075ms, p90: 0.086ms, p99: 0.106ms, p999: 0.259ms, max: 10.815ms
Current: hits: 12938, mean: 0.077ms, p50: 0.075ms, p90: 0.084ms, p99: 0.104ms, p999: 0.252ms, max: 10.751ms
Current: hits: 12853, mean: 0.077ms, p50: 0.075ms, p90: 0.085ms, p99: 0.108ms, p999: 0.250ms, max: 10.687ms
Current: hits: 12690, mean: 0.078ms, p50: 0.075ms, p90: 0.085ms, p99: 0.109ms, p999: 0.255ms, max: 11.007ms
Current: hits: 13056, mean: 0.076ms, p50: 0.075ms, p90: 0.084ms, p99: 0.105ms, p999: 0.190ms, max: 11.007ms
Current: hits: 12942, mean: 0.077ms, p50: 0.075ms, p90: 0.084ms, p99: 0.106ms, p999: 0.254ms, max: 10.879ms
Current: hits: 12970, mean: 0.076ms, p50: 0.075ms, p90: 0.084ms, p99: 0.105ms, p999: 0.242ms, max: 10.623ms
Current: hits: 12983, mean: 0.076ms, p50: 0.075ms, p90: 0.084ms, p99: 0.105ms, p999: 0.241ms, max: 11.007ms
Current: hits: 12801, mean: 0.077ms, p50: 0.075ms, p90: 0.085ms, p99: 0.108ms, p999: 0.253ms, max: 11.071ms
Current: hits: 12808, mean: 0.077ms, p50: 0.075ms, p90: 0.084ms, p99: 0.110ms, p999: 0.252ms, max: 20.991ms
============== Done! =================
Ok: hits: 767812, mean: 0.077ms, p50: 0.075ms, p90: 0.085ms, p99: 0.109ms, p999: 0.257ms, max: 21.631ms
OverLimit: hits: 0, mean: 0.000ms, p50: 0.000ms, p90: 0.000ms, p99: 0.000ms, p999: 0.000ms, max: 0.000ms
Unknown: hits: 0, mean: 0.000ms, p50: 0.000ms, p90: 0.000ms, p99: 0.000ms, p999: 0.000ms, max: 0.000ms
Overall: hits: 767812, mean: 0.077ms, p50: 0.075ms, p90: 0.085ms, p99: 0.109ms, p999: 0.257ms, max: 21.631ms
0.01ms | | 0.0th %-ile
0.02ms | | 0.0th %-ile
0.03ms | | 0.0th %-ile
0.04ms | | 0.0th %-ile
0.05ms | * | 0.0th %-ile
0.06ms | * | 0.3th %-ile
0.07ms | *************** | 18.4th %-ile
0.08ms | ************************************************ | 77.8th %-ile
0.09ms | ************** | 94.6th %-ile
0.10ms | *** | 98.1th %-ile
0.11ms | * | 99.0th %-ile
0.12ms | * | 99.4th %-ile
0.13ms | * | 99.6th %-ile
0.14ms | * | 99.7th %-ile
0.15ms | * | 99.7th %-ile
0.16ms | * | 99.7th %-ile
0.17ms | * | 99.7th %-ile
0.18ms | * | 99.8th %-ile
0.19ms | * | 99.8th %-ile
0.20ms | * | 99.8th %-ile
0.21ms | * | 99.8th %-ile
0.22ms | * | 99.8th %-ile
0.23ms | * | 99.8th %-ile
0.24ms | * | 99.9th %-ile
0.25ms | * | 99.9th %-ile
0.26ms | * | 99.9th %-ile
Latency is 0 RTT (jitter 1RTT for read and 2 RTT for read+write)
Adding this new test for the record: Enables rate limit response headers
limitador-server /bench/limits.yaml --rate-limit-headers DRAFT_VERSION_03 redis redis://ec2-34-224-88-175.compute-1.amazonaws.com:9876
root@cd25c8659ec6:/bench# /bench/limitador-driver rpc://127.0.0.1:8081 1m
Hitting Limitador at rpc://127.0.0.1:8081 for a total duration of 1 minutes
Current: hits: 47, mean: 21.394ms, p50: 21.503ms, p90: 21.631ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 47, mean: 21.519ms, p50: 21.631ms, p90: 21.759ms, p99: 22.015ms, p999: 22.015ms, max: 22.015ms
Current: hits: 47, mean: 21.347ms, p50: 21.375ms, p90: 21.631ms, p99: 22.015ms, p999: 22.015ms, max: 22.015ms
Current: hits: 47, mean: 21.407ms, p50: 21.503ms, p90: 21.759ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 47, mean: 21.323ms, p50: 21.375ms, p90: 21.631ms, p99: 22.015ms, p999: 22.015ms, max: 22.015ms
Current: hits: 47, mean: 21.304ms, p50: 21.375ms, p90: 21.631ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 47, mean: 21.391ms, p50: 21.503ms, p90: 21.759ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 47, mean: 21.494ms, p50: 21.631ms, p90: 21.759ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 47, mean: 21.274ms, p50: 21.247ms, p90: 21.503ms, p99: 21.631ms, p999: 21.631ms, max: 21.631ms
Current: hits: 47, mean: 21.489ms, p50: 21.503ms, p90: 21.887ms, p99: 22.015ms, p999: 22.015ms, max: 22.015ms
Current: hits: 47, mean: 21.345ms, p50: 21.375ms, p90: 21.631ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 47, mean: 21.451ms, p50: 21.503ms, p90: 21.759ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 47, mean: 21.317ms, p50: 21.375ms, p90: 21.759ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 47, mean: 21.399ms, p50: 21.503ms, p90: 21.631ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 47, mean: 21.320ms, p50: 21.375ms, p90: 21.631ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 48, mean: 21.184ms, p50: 21.247ms, p90: 21.503ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 47, mean: 21.323ms, p50: 21.375ms, p90: 21.631ms, p99: 22.271ms, p999: 22.271ms, max: 22.271ms
Current: hits: 48, mean: 21.219ms, p50: 21.247ms, p90: 21.631ms, p99: 22.143ms, p999: 22.143ms, max: 22.143ms
Current: hits: 48, mean: 21.240ms, p50: 21.247ms, p90: 21.631ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 48, mean: 21.173ms, p50: 21.247ms, p90: 21.503ms, p99: 22.271ms, p999: 22.271ms, max: 22.271ms
Current: hits: 48, mean: 21.256ms, p50: 21.375ms, p90: 21.631ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 48, mean: 21.173ms, p50: 21.247ms, p90: 21.503ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 47, mean: 21.285ms, p50: 21.375ms, p90: 21.503ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 48, mean: 21.037ms, p50: 21.119ms, p90: 21.375ms, p99: 21.503ms, p999: 21.503ms, max: 21.503ms
Current: hits: 48, mean: 21.149ms, p50: 21.247ms, p90: 21.503ms, p99: 21.631ms, p999: 21.631ms, max: 21.631ms
Current: hits: 48, mean: 21.195ms, p50: 21.247ms, p90: 21.503ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 47, mean: 21.339ms, p50: 21.375ms, p90: 21.631ms, p99: 22.015ms, p999: 22.015ms, max: 22.015ms
Current: hits: 47, mean: 21.380ms, p50: 21.375ms, p90: 21.759ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 47, mean: 21.364ms, p50: 21.375ms, p90: 21.887ms, p99: 22.015ms, p999: 22.015ms, max: 22.015ms
Current: hits: 47, mean: 21.516ms, p50: 21.631ms, p90: 21.759ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 47, mean: 21.388ms, p50: 21.503ms, p90: 21.759ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 47, mean: 21.328ms, p50: 21.375ms, p90: 21.759ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 48, mean: 21.224ms, p50: 21.247ms, p90: 21.503ms, p99: 21.631ms, p999: 21.631ms, max: 21.631ms
Current: hits: 48, mean: 21.264ms, p50: 21.247ms, p90: 21.631ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 47, mean: 21.279ms, p50: 21.375ms, p90: 21.631ms, p99: 22.015ms, p999: 22.015ms, max: 22.015ms
Current: hits: 47, mean: 21.500ms, p50: 21.631ms, p90: 21.759ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 47, mean: 21.380ms, p50: 21.503ms, p90: 21.759ms, p99: 22.143ms, p999: 22.143ms, max: 22.143ms
Current: hits: 47, mean: 21.326ms, p50: 21.375ms, p90: 21.759ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 48, mean: 21.160ms, p50: 21.247ms, p90: 21.503ms, p99: 21.631ms, p999: 21.631ms, max: 21.631ms
Current: hits: 48, mean: 21.168ms, p50: 21.247ms, p90: 21.503ms, p99: 21.631ms, p999: 21.631ms, max: 21.631ms
Current: hits: 48, mean: 21.109ms, p50: 21.119ms, p90: 21.503ms, p99: 21.631ms, p999: 21.631ms, max: 21.631ms
Current: hits: 47, mean: 21.353ms, p50: 21.375ms, p90: 21.759ms, p99: 22.015ms, p999: 22.015ms, max: 22.015ms
Current: hits: 47, mean: 21.296ms, p50: 21.375ms, p90: 21.631ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 47, mean: 21.494ms, p50: 21.631ms, p90: 21.887ms, p99: 22.015ms, p999: 22.015ms, max: 22.015ms
Current: hits: 47, mean: 21.377ms, p50: 21.375ms, p90: 21.759ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 47, mean: 21.383ms, p50: 21.503ms, p90: 21.759ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 48, mean: 21.192ms, p50: 21.247ms, p90: 21.503ms, p99: 21.631ms, p999: 21.631ms, max: 21.631ms
Current: hits: 48, mean: 21.243ms, p50: 21.247ms, p90: 21.631ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 47, mean: 21.369ms, p50: 21.375ms, p90: 21.759ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 48, mean: 21.269ms, p50: 21.375ms, p90: 21.631ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 47, mean: 21.296ms, p50: 21.375ms, p90: 21.631ms, p99: 21.887ms, p999: 21.887ms, max: 21.887ms
Current: hits: 48, mean: 21.259ms, p50: 21.375ms, p90: 21.503ms, p99: 21.631ms, p999: 21.631ms, max: 21.631ms
Current: hits: 48, mean: 21.125ms, p50: 21.247ms, p90: 21.503ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 48, mean: 21.237ms, p50: 21.247ms, p90: 21.503ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 48, mean: 21.099ms, p50: 21.119ms, p90: 21.503ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 48, mean: 21.149ms, p50: 21.247ms, p90: 21.375ms, p99: 22.015ms, p999: 22.015ms, max: 22.015ms
Current: hits: 48, mean: 21.248ms, p50: 21.375ms, p90: 21.503ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 48, mean: 21.240ms, p50: 21.375ms, p90: 21.503ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
Current: hits: 48, mean: 21.248ms, p50: 21.375ms, p90: 21.503ms, p99: 21.759ms, p999: 21.759ms, max: 21.759ms
============== Done! =================
Ok: hits: 2818, mean: 21.295ms, p50: 21.375ms, p90: 21.631ms, p99: 21.887ms, p999: 22.143ms, max: 22.271ms
OverLimit: hits: 0, mean: 0.000ms, p50: 0.000ms, p90: 0.000ms, p99: 0.000ms, p999: 0.000ms, max: 0.000ms
Unknown: hits: 0, mean: 0.000ms, p50: 0.000ms, p90: 0.000ms, p99: 0.000ms, p999: 0.000ms, max: 0.000ms
Overall: hits: 2818, mean: 21.295ms, p50: 21.375ms, p90: 21.631ms, p99: 21.887ms, p999: 22.143ms, max: 22.271ms
2.19ms | | 0.0th %-ile
4.38ms | | 0.0th %-ile
6.56ms | | 0.0th %-ile
8.75ms | | 0.0th %-ile
10.94ms | | 0.0th %-ile
13.13ms | | 0.0th %-ile
15.32ms | | 0.0th %-ile
17.50ms | | 0.0th %-ile
19.69ms | | 0.0th %-ile
21.88ms | ******************************************************************************** | 99.5th %-ile
24.07ms | * | 100.0th %-ile
I do not notice any difference regarding the first test without the rate limit response headers
Test setup
Redis proxy
1 EC2 instance
us-east-1 (N. Virginia)
AZ: us-east-1a
amazon linux 2023 AMI
type: c7i.large 2vCPU 4Gib Memory
Elasticache
Limtador and Driver
1 EC2 instance
eu-east-2 (Ohio)
AZ: eu-north-1c
amazon linux 2023 AMI
type: c7i xlarge 4vCPU 8Gib Memory
Limits designed to never hit the threshold, so all requests increment the counter.
Results