[v5] Test result ignores JUnit rules

[talmoshe24]

I'm using RP 5.3.3 and agent-java-junit v4 5.0-RC I have a case where I'm using TestRule and the test is reported as a fail, although it passes

Retry rule class:

public class Retry implements TestRule {

    private static final Logger logger = LogManager.getLogger(Retry.class);

    private int retryCount;

    public Retry(int retryCount) {
        this.retryCount = retryCount;
    }

    public Statement apply(Statement base, Description description) {
        return statement(base, description);
    }

    private Statement statement(final Statement base, final Description description) {
        return new Statement() {
            @Override
            public void evaluate() throws Throwable {
                Throwable caughtThrowable = null;

                // implement retry logic here
                for (int i = 0; i < retryCount; i++) {
                    try {
                        base.evaluate();
                        return;
                    } catch (Throwable t) {
                        caughtThrowable = t;
                        logger.warn(description.getDisplayName() + ": run " + (i+1) + " failed");
                    }
                }
                logger.error(description.getDisplayName() + ": giving up after " + retryCount + " failures");
                throw caughtThrowable;
            }
        };
    }
}

In my test class, I have the implementation:

    @Rule
    public TestRule retry = new Retry(3);

When there is an exception in one of the tests once, the RP reports that the test is failed although it passed in the second time

[HardNorth]

@talmoshe24 The only supported way of retrying is that is provided by JUnit Foundation. Otherwise our agent have no clue whether it's a retry or not. If you implement a custom rule you also have to implement handling logic by yourself.