Issue with attaching JUnit RunListener in Gradle Project

[cah-daniel-adams01]

Hi @sbabcoc , I am having issues with attempting to attach my JUnit RunListener in my gradle project. I have followed all of the steps outlined in the article here -> https://discuss.gradle.org/t/how-to-attach-a-runlistener-to-your-junit-4-tests-in-gradle/30788 and my project is still not picking up the RunListener. Any help would be appreciated, some project specs:

• Geb/Spock Java/Groovy project
• Headless Chrome tests
• JUnit 4
• Attempting to attach a the com.adaptavist.tm4j.junit.ExecutionListener provided by TM4J for JIRA

Specific issue is that there is no indication that the RunListener is being picked up at execution time, and no contextual logging is being output that would indicate errors/misuse

[sbabcoc]

What does your ServiceLoader configuration file look like? What is its path within your project, and what does it contain?

This file should be at src/test/resources/META-INF/services/org.junit.runner.notification.RunListener

This file should contain a single line:

com.adaptavist.tm4j.junit.ExecutionListener 

[cah-daniel-adams01]

Yes, here is the path etc.

[sbabcoc]

Your configuration file looks correct. What does your build.gradle file look like? Are all of the element included? What happens if you misspell the class name of the listener class in the ServiceLoader configuration file?

To install the hooks needed to publish test lifecycle notifications, JUnit Foundation uses Byte Buddy to intercept key methods of the JUnit 4 API. This process requires that none of the related JUnit classes are loaded beforehand.

[cah-daniel-adams01]

I cannot include my entire build.gradle file due to company policies etc. When I misspell the classname in ServiceLoader, nothing happens, no errors. All build.gradle configurations defined in https://discuss.gradle.org/t/how-to-attach-a-runlistener-to-your-junit-4-tests-in-gradle/30788 are included. Here is the segment of my build.gradle including and following the dependencies block

dependencies {
    // If using Spock, need to depend on geb-spock
    compile 'org.ysb33r.gradle:grolifant:0.5.1'
    compile 'com.nordstrom.tools:junit-foundation:9.1.1'
    compile group: 'org.slf4j', name: 'slf4j-log4j12', version: '1.7.25'
    compile 'org.testng:testng:6.8.8'
    testCompile "org.gebish:geb-spock:$gebVersion"
    testCompile("org.spockframework:spock-core:1.1-groovy-2.4") {
        exclude group: "org.codehaus.groovy"
    }
    testCompile "org.codehaus.groovy:groovy-all:$groovyVersion"

    // If using JUnit, need to depend on geb-junit (3 or 4)
    testCompile "org.gebish:geb-junit4:$gebVersion"
    testCompile 'com.aoe:geb-spock-reports:0.2.3'

    // Drivers
    testCompile "org.seleniumhq.selenium:selenium-chrome-driver:$seleniumVersion"
    testCompile "org.seleniumhq.selenium:selenium-support:$seleniumVersion"
    testCompile "org.seleniumhq.selenium:selenium-java:$seleniumVersion"

    testCompile 'org.apache.poi:poi:3.12'
    testCompile 'org.apache.poi:poi-ooxml:3.14'
    testCompile 'org.apache.poi:openxml4j:1.0-beta'
    //testCompile "org.seleniumhq.selenium:selenium-firefox-driver:$seleniumVersion"

    testCompile 'net.sourceforge.jexcelapi:jxl:2.6.12'
    testCompile 'org.codehaus.jackson:jackson-mapper-asl:1.9.13'
    testCompile( 'com.athaydes:spock-reports:1.6.2' ) {
        transitive = false // this avoids affecting your version of Groovy/Spock
    }
    testCompile 'org.slf4j:slf4j-api:1.7.13'
    testCompile 'org.slf4j:slf4j-simple:1.7.13'
    // https://mvnrepository.com/artifact/com.adaptavist/tm4j-junit-integration
    compile group: 'com.adaptavist', name: 'tm4j-junit-integration', version: '1.0.0'

}

webdriverBinaries {
    chromedriver chromeDriverVersion
    geckodriver geckoDriverVersion
}

ext{
    junitFoundation = configurations.compile.resolvedConfiguration.resolvedArtifacts.find { it.name == 'junit-foundation' }
}

baseUrls.each { entry ->
    drivers.each { driver ->
        task "${entry.name}-${driver}Test"(type: Test) {
            group JavaBasePlugin.VERIFICATION_GROUP

            outputs.upToDateWhen { false }  // Always run tests

            systemProperty "geb.build.reportsDir", reporting.file("geb/$name")
            systemProperty "geb.env", driver
            systemProperty "geb.build.baseUrl", entry.url
            systemProperty 'spock.configuration', 'RemoteSpockConfig.groovy'
        }
    }
}

test {
    dependsOn drivers.collect { baseUrls.each{ entry -> tasks["${entry.name}-${it}Test"]} }
    enabled = false
}

tasks.withType(Test) {
    println junitFoundation.file
    jvmArgs "-javaagent:${junitFoundation.file}"
    maxHeapSize = "1g"
    jvmArgs '-XX:MaxMetaspaceSize=128m'
    systemProperties System.properties
    testLogging.showStandardStreams = true
    testLogging {
        exceptionFormat = 'full'
    }
    reports {
        junitXml.enabled = true
        html.enabled = false
    }
}

tasks.withType(GroovyCompile) {
    groovyOptions.forkOptions.memoryMaximumSize = '256m'
}

[sbabcoc]

Does your println statement execute and show the correct path to the JAR? I was just doing some debugging through the execution and discovered that the finished method isn't being intercepted. This issue is apparently related to the work I did to support Java 7, and it shouldn't affect the behavior of a standard RunListener.

[cah-daniel-adams01]

Yes, it results in.....

....\junit-foundation-9.1.1.jar

If you want me to add any other debugging statements I can do so.

[sbabcoc]

A misspelled provider class name should result in an exception:

java.util.ServiceConfigurationError: org.junit.runner.notification.RunListener: Provider com.nordstrom.automation.junit.RunAnnounce not found

This would occur in the context of org.junit.runners.ParentRunner#run(RunNotifier) The fact that you're not seeing the configuration exception strongly suggests that the ParentRunner class is being loaded into the class cache before JUnit Foundation can install its method interceptor.

[sbabcoc]

Upgrade JUnit Foundation to version 9.4.3 and see if you get any test lifecycle log messages on the console. You might also consider upgrading TestNG to version 6.10... although it's unusual for a project to include both JUnit and TestNG. (For JUnit Foundation, I drive the unit tests with TestNG.) If you don't see any console output from JUnit Foundation, this suggests that Gradle isn't actually running the Java agent. The documentation on jvmArgs is very sketchy, and looking thru the implementation makes me suspect that order and structure of declarations may affect how the args are processed.

[cah-daniel-adams01]

After adding both suggested modifications, I am still seeing no log messages from JUnit Foundation. Do you have an example of log messages that I should be seeing?

[sbabcoc]

    12:04:11.564 [Test worker] DEBUG com.nordstrom.automation.junit.Run - runStarted: org.junit.runners.BlockJUnit4ClassRunner@1f424818
    12:04:13.087 [Test worker] DEBUG c.n.a.junit.RunReflectiveCall - beforeInvocation: beforeClass(com.epam.reportportal.junit.DummyTest)
    12:04:13.192 [Test worker] DEBUG c.n.a.junit.RunReflectiveCall - afterInvocation: beforeClass(com.epam.reportportal.junit.DummyTest)
    12:07:12.626 [Test worker] WARN  c.n.automation.junit.JUnitConfig - Unable to locate configuration at path 'junit.properties'

These are the sorts of lifecycle message you'll see in the console output when the interceptors are connected and working as they should. All loggers are under package com.nordstrom.automation.junit.

[cah-daniel-adams01]

Unfortunately not seeing any of these messages when I run the tests. Closing this issue due to inability to share anymore project information details as they cannot be public. If this issue is truly the same, I'm sure others will post similar issues.

[sbabcoc]

Seems like the Java agent isn't actually being run. I'm not a Gradle expert by anyone's estimation. Perhaps this is a Gradle issue after all.

[sbabcoc]

Try removing the second statement that sets jvmArgs to see if you get a different result. It's even possible that the reports reference could be having an effect.

[sbabcoc]

Another thing to try would be to alter the JAR path to see if you get a failure. This would at least determine whether or not the Java agent is actually being run.

[mformetal]

I'm having a similar issue, with a complicated-ish setup:

Repository A consumes a composite-build that contains a "shadowed" version of the junit_foundation jar. It then adds a dependency on that included build like the Gradle configuration conveys. I see the -javaagent in my jvmargs when I run a build with --info, but I'm not seeing those console logs.

Repository B consumes this same composite build, I'm seeing the -javaagent string there, but the console logs are printed.

Any idea of where I should look first? (Sorry to necro dead issue but saw your comments and figured it may be useful for someone else)

UPDATE: The agent seems to be configured and running, but none of the reflection interception magic is working - must be due to some local config. Sorry for the confusion!