Closed alim-akbashev closed 3 years ago
Can you try version attached?
thank you for a quick response.
i've deployed service with version you've provided, collecting metrics now, will be back with feedback tomorrow
as I can see, the issue still there:
by checking values in redis, it seems that after some time a cardinality of sorted set ...:permits
stops matching the value ...:value
Did you make sure that all Redisson instances have synced clock?
not sure about nanoseconds precision, but i'm sure clocks are pretty in sync. all instances are running in the same k8s cluster (amazon eks), all k8s nodes are synced.
btw, not sure if it's important, my redis is a managed one - single AWS ElastiCache running on m6g.large (64bit arm Graviton).
let me try to build more reproducible sandbox
it's more or less reproducible. Please check the snippet https://gist.github.com/alim-akbashev/316f56197f099f323ff68cd27b51df78 It's more simple to reproduce dritft when numbers in config are big, like 30k requests per 30 seconds, but issue appears with lower values as well, it just require more time to drift.
just in case, my config i've used to run that snippet: just regular macbook pro (Intel), Redis 6.2.5 in docker, openjdk 16
p.s. while i've been typing this comment, limit has dropped down to 29494, so it takes minutes, not hours with such config
Fixed! Thanks for report
Rate limit decreases over the time in highly concurrent environment. Demo of such drift is on screenshot. Rate there has been limited by only RRateLimiter.
Expected behavior Limit should not change without explicit call of setRate() with new value.
Actual behavior Limit changes over hours. As a workaround I explicitly call setRate() with the same values time to time.
Steps to reproduce or test case Just regular usage, nothing special:
Redis version 6.0.5
Redisson version 3.16.1
Redisson configuration