gradle / test-retry-gradle-plugin

Gradle plugin to retry tests that have failed to mitigate test flakiness.
Apache License 2.0
218 stars 48 forks source link

"The following test methods could not be retried, which is unexpected" for the test method with parentheses in its name #263

Closed givehighfive closed 3 months ago

givehighfive commented 4 months ago

Parentheses in the test name are legit from Kotlin+Gradle+JUnit5 perspective, but it's caused exception in test-retry-gradle-plugin:

build.gradle.kts:

plugins {
    kotlin("jvm") version "1.9.22"
    id("org.gradle.test-retry") version "1.5.8"
}

group = "com.example.test"
version = "1.0-SNAPSHOT"

repositories {
    mavenCentral()
}

dependencies {
    testImplementation("org.jetbrains.kotlin:kotlin-test")
}

tasks.test {
    useJUnitPlatform()
}
kotlin {
    jvmToolchain(17)
}

tasks.test {
    retry {
        maxRetries = 2
        maxFailures = 3
        failOnPassedAfterRetry = false
    }
}

Test class:

import org.junit.jupiter.api.Test

class DemoTest {

    @Test
    fun `retries work for this test`() {
        throw AssertionError()
    }

    @Test
    fun `and retries should work for this test (but not)`() {
        throw AssertionError()
    }
}

Output:

The following test methods could not be retried, which is unexpected. Please file a bug report at https://github.com/gradle/test-retry-gradle-plugin/issues
     DemoTest#and retries should work for this test (but not)()

The same reproduced with gradle-enterprise-gradle-plugin:3.16.1

jean-andre-gauthier commented 3 months ago

This has been fixed in https://github.com/gradle/test-retry-gradle-plugin/pull/272, and will be released in 1.5.8