search

gradle / test-retry-gradle-plugin

Gradle plugin to retry tests that have failed to mitigate test flakiness.
Apache License 2.0
220 stars 49 forks source link

Test retry on class level fails with suite engine and Gradle 5.0 #230

Closed snoopcheri closed 10 months ago

snoopcheri commented 10 months ago

When using the Test Retry Gradle plugin with Gradle 5.0 and using the JUnit Suite engine, retrying tests on class level fails.

TestSuite
  Test1
    testOk()
    testFlaky() // fails on 1st execution and passes on 2nd
  Test2
    testOk()
    testFlaky() // fails on 1st execution and passes on 2nd

The class Test1 is configured to be retried on class-level.

When running this with Gradle 5.0, the following error is shown:

The following test methods could not be retried, which is unexpected.

No test methods are listed though.

The reason for this problem is the fact that Gradle 5.0 does not report test events for the intermediate classes Test1 and Test2. Only events for TestSuite and the test methods are reported.