Closed oguzhanunlu closed 3 years ago
What I meant is that the backoff strategy should be applied only on faulty pipelines, i.e. if you see a throughput exception, in normal circumstances it would harm more (delay the job).
Another point is that if we're going to use a backoff period - it should be a reverse backoff, i.e. 3 min, 1 min, 30 sec, 20 sec. It reduces the overall amount of calls, but also makes sure the job isn't stuck.
I think this is what we need to do.
Continuing at #69
Thanks for the quick review @chuwy , I also wasn't sure about the strategy.
Yeah it makes sense, first hit of the rate limit ex can be followed by the backoff strategy.
On the strategy, how about min (30 sec) - max (1 min) - factor (1.1) and jitter enabled so that we have 30 sec > 33 sec > 36.3 sec and so on? Compared to constant 30 sec, factor of 1.1 is a bit more polite. What do you think?
I'll check if I can do this without reinventing the wheel