search

resilience4j / resilience4j

Resilience4j is a fault tolerance library designed for Java8 and functional programming
Apache License 2.0
9.68k stars 1.33k forks source link

Implement Hedged Requests #1454

Open duke-arioch opened 3 years ago

duke-arioch commented 3 years ago

I was reading about the "Hedged Requests" pattern and thought r4j might be a good place to create an implementation.

The idea is to start dealing with long-tail requests. To do so, you would need to track such requests at some cutoff, for example, 95%. If a task took longer than some minimum amount of time and took longer than 95% of other requests, n extra parallel requests would be issued. The first one to come back would cancel all the rest. This can cause a huge reduction in response time where there is a small chance of back-end tail latency.

RobWin commented 3 years ago

Interesting idea. Do you plan to implement something like this for your application?

duke-arioch commented 3 years ago

I just came across it in Go, which I am expanding into a bit. Most of my applications are Java, and yes - for one mission-critical app that benefits from quick response - I am considering implementing it. The application currently uses the resilience4j library.

Depending on the priority here, and my progress at implementing, I would be happy to contribute whatever I put together.

The concept fits well with the established patterns here. My concerns would be how rigorously I want to calculate the metrics for the behavior, and how it behaves during catastrophic failures.

RobWin commented 3 years ago

If you are starting to implement something, just create a PR and describe your proposed solution. I'm happy to help and answer questions.

duke-arioch commented 3 years ago

just raised PR #1490. Looking for comments etc. There may be a rough edge or two on javadoc, test coverage and performance testing and I am sure other areas as well. If I can get to something acceptable I will look for recommendations on adding support for different frameworks, other metrics types (saw some interesting work on dynamic quantiles) and other decorators.

duke-arioch commented 3 years ago

I'd like to have a more detailed discussion on this. Can I get added to the Slack?

RobWin commented 3 years ago

Which email address should I use to invite you?

duke-arioch commented 3 years ago

matthew.sandoz@gmail.com is fine.