mergeReruns = true is not working as expected

[rekha1211]

I have a project with Junit5 Version -5.3.2 Gradle Version - 6.8.1 Jenkins Junit plugin- latest Selenide - 4.3 Retry Plugin Version - 1.4.0

I tried implementing the mergeReruns concept in my build.gradle and it does not work as expected.

Expected: If a test failed the first time and passed the second time it should not display the test name is results Actual: It is still displaying the test name is Jenkins test results

Expected: If a test fails the both times it should display the test name is failed test results. Actual: It is not displaying the test name is Jenkins failed test results

Some tests results show right and some do not.

Below are some approaches that I tried in my build.gradle configuration. Can I know if I am doing anything wrong or if there is version compatibility issue .

Approach 1 has the whole build.gradle syntax. Approach 2 and 3 has the basic syntax chnages that I made regarding the retry plugin.

Approach 1:

plugins {

    id 'org.gradle.test-retry' version '1.4.0'
}

apply plugin: 'org.gradle.test-retry'

test {
    retry {
        failOnPassedAfterRetry = false
        maxFailures = 75
        maxRetries = 1
        useJUnitPlatform() {

        includeTags 'Registration'
        includeTags 'Search'
        }
    }

    // set heap size for the test JVM(s)
    minHeapSize = "1024m"
    maxHeapSize = "4000m"

    def date = new Date()
    def formattedDate = date.format('yyyy-MM-dd')
    def formattedTime = date.format('HH-mm')
    def htmlTestReport = "$buildDir\\reports\\" + formattedDate + "\\" + formattedTime
    //def xmlTestReport = "$buildDir\\test-reports\\" + formattedDate + "\\" + formattedTime
    def xmlTestReport = "$buildDir\\test-reports\\"

    if (project.hasProperty('buildNumber')) {
        htmlTestReport = "$buildDir\\screenshots\\" + project.getProperty('buildNumber').toString()
    }

    environment "FOLDER_NAME", htmlTestReport

    reports {
        junitXml {
            outputPerTestCase = true // defaults to false
            mergeReruns = true // defaults to false
        }
        junitXml.enabled = true

        html.enabled = project.hasProperty('doHTML') ? project.getProperty('doHTML').toBoolean() : true
        //use -PdoHTML=false to *not* run the HTMl report
        println("XML report destination path : " + xmlTestReport)
        junitXml.destination = file(System.getenv('REPORT_PATH') ?: xmlTestReport)
        html.destination = file(System.getenv('REPORT_PATH') ?: htmlTestReport)
    }
    testLogging {
        events "passed", "skipped", "failed"
        showStandardStreams = true
    }
    // can add "started" to the above list for more logging
    ignoreFailures = true
}

testlogger {
    theme 'standard-parallel'
    slowThreshold 3 * 60 * 1000 // 3 Minutes in Milliseconds
}

Approach 2: I even tried some other syntax as below

test {
        useJUnitPlatform() {
        includeTags 'Registration'
        includeTags 'Search'
        }
 retry {
        failOnPassedAfterRetry = false
        maxFailures = 75
        maxRetries = 1
}
}

reports {
        junitXml {
            outputPerTestCase = true // defaults to false
            mergeReruns = true // defaults to false
        }
    }

Approach3:

test {
        useJUnitPlatform() {
        includeTags 'Registration'
        includeTags 'Search'
        }
 retry {
        failOnPassedAfterRetry = false
        maxFailures = 75
        maxRetries = 1
}
}
reports.junitXml.mergeReruns = true
reports {
        junitXml {
            outputPerTestCase = true // defaults to false
        }
    }

[marcphilipp]

@rekha1211 Could you please provide a full minimal reproducer?

[pshevche]

I am afraid we won't be able to make progress without a reproducer. The reporting configuration is very helpful, but we will also need an example test, for which the report merge fails.