Betterment / delayed

a multi-threaded, SQL-driven ActiveJob backend used at Betterment to process millions of background jobs per day
MIT License
156 stars 9 forks source link

Consider adding jitter to retry interval #26

Open valerauko opened 1 year ago

valerauko commented 1 year ago

In the case of many jobs getting enqueued at about the same time, they might overwhelm the system, fail and be retried at some later point.

However if there's so many and they all get retried at the same (exponential backoff) interval, they might still end up causing trouble.

Adding a(n option to add a) random jitter to the retry interval might distribute the load a bit better and resolve the retry stampede.