search

reportportal / agent-java-junit

Report Portal agent for JUnit 4
Apache License 2.0
11 stars 17 forks source link

agent-java-junit4 does NOT report test results being run with @RunWith(PowerMockRunner.class) annotation #66

Closed sspieker-cc closed 3 years ago

sspieker-cc commented 4 years ago

When executing test suites using any runner OTHER than PowerMockRunner are reported correctly. However, when PowerMockRunner is utilized, those tests classes are fail to report any results at all to ReportPortal.

ReportPortal installation is version 5 GA.

Agent-java-junit4 dependencies for all projects are:

com.epam.reportportal:agent-java-junit:4.1.8
com.epam.reportportal:logger-java-log4j:4.0.1
com.epam.reportportal:logger-java-logback:4.0.4
com.epam.reportportal:client-java:4.0.12
com.epam.reportportal:commons-model:4.2.0
com.nordstrom.tools:junit-foundation:9.2.0
com.google.inject:guice:4.2.2
io.reactivex.rxjava2:rxjava:2.1.6
org.apache.logging.log4j:log4j-api:2.10.0
org.apache.logging.log4j:log4j-core:2.10.0

The runner has been confirmed as the issue as one gradle project did not utilize the powermock api and did not report the results for the three methods contained within. This project was thus updated and so the runner was switched to utilize the @RunWith(MockJUnitRunner.class) and the results from this test class has since been reporting all test results correctly to ReportPortal.

Projects utilize a variety of mockito as well as PowerMock to execute tests. The hope here is that the agent will work with the PowerMockRunner class as well as the others.

Agent-java-junit v5.0.0-BETA-8 could not be tested due to the snapshot compile dependencies for commons-model and client-java.

If these dependencies can be supplied, I will be happy to attempt a run using the updated versions.

Thanks, Scott

sspieker-cc commented 4 years ago

After updating the agent-java-junit client to version 5.0.0-BETA-8 (and fudging all of the dependency references within the local repository) I have confirmed that the PowerMockRunner continues to be ignored by the agent tooling.

HardNorth commented 4 years ago

Can you try agent-java-junit:5.0.0-BETA-9?

HardNorth commented 3 years ago

It looks like we can't do anything with that, since in this case our upstream library JUnit Foundation is not calling our listener at all.

HardNorth commented 3 years ago

An issue on JUnit Foundation: https://github.com/sbabcoc/JUnit-Foundation/issues/77

HardNorth commented 3 years ago

So for those who still want to use PowerMock you must:

  1. use at least agent version 5.0.0: https://github.com/reportportal/agent-java-junit/releases/tag/5.0.0
  2. Add @PowerMockRunnerDelegate(BlockJUnit4ClassRunner.class) to your test classes as here: https://github.com/reportportal/agent-java-junit/blob/develop/src/test/java/com/epam/reportportal/junit/features/powermock/SimplePowermockTest.java

Or it is better to migrate on Mockito or any other framework, since Powermock is too outdated (got last update in 2016) and it's not compatible with Java 11+