search

dotnet / extensions

This repository contains a suite of libraries that provide facilities commonly needed when creating production-ready applications.
MIT License
2.55k stars 731 forks source link

Standard resilience pipeline: shouldn't "Total request timeout" policy come before "Rate limiter" policy? #5185

Open tyler-boyd opened 1 month ago

tyler-boyd commented 1 month ago

re: https://github.com/dotnet/extensions/issues/5078#issuecomment-2133515538

I would love to see an explanation for why the policies are there in the order they are in.

As an example, why is the total request timeout policy inside/after the rate limit policy, when the rate limit policy can cause large delays if you enable queueing? For example...if you have a queue size of 10 and parallelism 1 and total request timeout = 30s, in the worst case scenario it could take that 10th queued request up to 300s, which is 10x higher than the total request timeout.

This is either bad or I'm misunderstanding something, and it would be much easier to tell which if the motivation behind the standard policies were shared.

I would have expected the total request timeout to be able to cancel a queued rate limit policy, but either deliberately or accidentally, this doesn't appear to be the case. Alternatively, perhaps a way to configure a "max queueing time" on the rate limit policy would help?

Thanks in advance!