failsafe-lib / failsafe

Fault tolerance and resilience patterns for the JVM
https://failsafe.dev
Apache License 2.0
4.16k stars 295 forks source link

Nested retries, how to prevent duplicate retrying? #379

Open pandoras-toolbox opened 5 months ago

pandoras-toolbox commented 5 months ago

I wonder how I can do something with Failsafe.

If methods which do a retry with Failsafe are nested then how can I prevent in the outter method that a retry is performed when an exception is thrown after the last retry of a inner method?

I would like to use the same Exception type in both aaa() and in ddd(), AssertionError.class.

public static void aaa() {
    Failsafe.with(RetryPolicy.builder()
            .withMaxRetries(1)
            .handle(AssertionError.class)
            .build())
        .run(() -> {
            System.out.println("aaa");
            bbb();
        });
}

public static void bbb() {
    System.out.println("bbb");
    ccc();
    ddd();
}

public static void ccc() {
    System.out.println("ccc");
}

public static void ddd() {
    Failsafe.with(RetryPolicy.builder()
            .withMaxRetries(1)
            .handle(AssertionError.class)
            .build())
        .run(() -> {
            System.out.println("ddd");
            throw new AssertionError("ddd");
        });
}

public static void main(String[] args) {
    MyClass.aaa();
    MyClass.bbb();
}

The console output should be:

But it is:

I guess it could be done if there would be something like a marker or so, to flag the exception thrown by a inner retry method so that it if the flag is present it does not do a retry in the outter method, even if the exception types match.

pandoras-toolbox commented 5 months ago

I now came up with something like:

public static void aaa() {
    Failsafe.with(RetryPolicy.builder()
            .withMaxRetries(1)
            .handleIf((o, throwable) -> throwable instanceof AssertionError && (throwable.getMessage() == null
                || !throwable.getMessage().contains("ddd")))
            .build())
        .run(() -> {
            System.out.println("aaa");
            bbb();
        });
}

But I do not know if that is the most elegant way to do that.

Tembrel commented 5 months ago

I don't understand what behavior you're trying to achieve. You want aaa(), in general, to retry some code if an AssertionError is thrown during the execution of that code.

But you don't want it to retry if a particular method that aaa() calls, ddd(), throws an AssertionError due to multiple failed attempts all throwing AssertionError.

Is the idea that an AssertionError thrown from ddd() is somehow special, meaning that there's no need for aaa() to handle it the way it would if it wasn't coming from a Failsafe call?

These are weird semantics, very non-modular. And trying handle AssertionErrors at all feels dangerous.

But assuming that's really what you want, then you could convert the AssertionError thrown by ddd() to some other known exception or error to avoid it being handled by aaa(), and then (if you want) convert it back to an AssertionError.

@SuppressWarnings("unchecked")
public static void aaa() {
    try {
    Failsafe.with(retryAssertionErrorOnce).run(() -> {
        System.out.println("aaa");
        bbb();
    });
    } catch (FailsafeException ex) {
    if (ex.getCause() instanceof AssertionError) throw (AssertionError) ex.getCause();
    throw ex;
    }
}

public static void bbb() {
    System.out.println("bbb");
    ccc();
    ddd();
}

public static void ccc() {
    System.out.println("ccc");
}

@SuppressWarnings("unchecked")
public static void ddd() {
    Failsafe.with(wrapAssertionError, retryAssertionErrorOnce).run(() -> {
    System.out.println("ddd");
    throw new AssertionError("ddd");
    });
}

static RetryPolicy retryAssertionErrorOnce =  RetryPolicy.builder()
    .withMaxRetries(1)
    .handle(AssertionError.class)
    .build();

static Fallback wrapAssertionError =
    Fallback.builderOfException(e -> new FailsafeException(e.getLastException()))
    .handle(AssertionError.class)
    .build();

public static void main(String[] args) {
    MyClass.aaa();
    MyClass.bbb();
}

This produces the output you were hoping for, but again, this all seems highly suspect to begin with.

pandoras-toolbox commented 5 months ago

Thank you, I was not aware of the possibility which you have shown in your example.

What I was looking was a kind of "circuit breaker" for retries I guess.

Lets assume there are nested retries. It does not matter what they do for the example. If a retry fails in one of the nested retry methods then, depending on the concrete case, it might be useless if the outer retry methods perform a retry on failure, because it would start the inner retries all over again, but it would be useless.

That can be handled with different exception types, but I was looking for a more elegant and safe way. Like if a retry policy can be configured to signal outer retry methods not to retry if they catch this signal. I do not know the Failsafe framework so much, but maybe if it would then throw a "FailsafeDoNotRetryException", then the outer retry blocks would not attempt to retry but fail instantly.

I am not sure if that makes sense. If not, no need to waste your time on that. What I wanted to achieve I could do now.

Tembrel commented 5 months ago

Like if a retry policy can be configured to signal outer retry methods not to retry if they catch this signal. I do not know the Failsafe framework so much, but maybe if it would then throw a "FailsafeDoNotRetryException", then the outer retry blocks would not attempt to retry but fail instantly.

You don't need a specialized exception type for that effect: Just throw an exception that isn't handled by any of outer Failsafe executors. That's what the code I presented above does, except I added code to translate that exception back to the original type. It's simpler without that code:

public static void aaa() {
    Failsafe.with(retryAssertionErrorOnce).run(() -> {
    System.out.println("aaa");
    bbb();
    });
}

public static void bbb() {
    System.out.println("bbb");
    ccc();
    ddd();
}

public static void ccc() {
    System.out.println("ccc");
}

public static void ddd() {
    Failsafe.with(wrapAssertionError, retryAssertionErrorOnce).run(() -> {
    System.out.println("ddd");
    throw new AssertionError("ddd");
    });
}

public static void main(String[] args) {
    MyClass.aaa();
    MyClass.bbb();
}

static RetryPolicy<Void> retryAssertionErrorOnce =  RetryPolicy.<Void>builder()
    .withMaxRetries(1)
    .handle(AssertionError.class)
    .build();

static Fallback<Void> wrapAssertionError =
    Fallback.<Void>builderOfException(e -> new FailsafeException(e.getLastException()))
    .handle(AssertionError.class)
    .build();
pandoras-toolbox commented 5 months ago

Thank you.