search

sbabcoc / JUnit-Foundation

JUnit Foundation is a lightweight collection of JUnit watchers, interfaces, and static utility classes that supplement and augment the functionality provided by the JUnit API.
Apache License 2.0
22 stars 6 forks source link

Version 16 does not support JunitParams library #122

Closed HardNorth closed 2 years ago

HardNorth commented 2 years ago

Hello there.

I've tried to update our library on JUnit-Foundation version 16 and it look like it does not support JunitParams library anymore. All my tests related to this functionality fail. From what I can see no testStarted and beforeInvocation calls for such tests.

sbabcoc commented 2 years ago

Hmmm... The unit test collection includes a couple of JUnitParams tests, and they appear to run just fine. The big difference with release 16+ is that I replaced a bunch of reflective operations with byte code transformations. If any of the core JUnit classes get loaded prior to the execution of JUnit Foundation agent, these transformations will be disabled. I haven't been able to run agent-java-junit tests for a while, because I lost access to a Report Portal server to test against.

There are also a couple of JUnitParams classes I'm transforming: junitparams.internal.ParameterisedTestMethodRunner and junitparams.internal.TestMethod. I wonder if these are the source of the issue here.

For the complete set of all transformed classes, take a look at this: https://github.com/sbabcoc/JUnit-Foundation/blob/master/src/main/java/com/nordstrom/automation/junit/JUnitAgent.java

sbabcoc commented 2 years ago

I've been unable to import the project into Eclipse, so I can't do any debugging. I'm not well-versed in the ways of Gradle and don't have a clue how to fix the fundamental issues I'm running into. I wonder if using JUnit 5 as the test harness may be a problem, though, since it might be referring to JUnit 4 classes before the JUnit Foundation agent has applied its byte code modifications. The unit tests in JUnit Foundation are driven via TestNG, which avoids these sorts of conflicts.

sbabcoc commented 2 years ago

Found the issue, coded a fix, and published a new release.

The issue was that I've overridden the getAnnotations method, caching the results so I don't have to regenerate them. Unbeknownst to me, the hashCode method of the java.lang.reflect.Method class is basically useless, producing the same result for methods with the same names but different parameters signatures. As a result of this core Java defect, test methods could get ignored because their annotations would not be returned.

The solution was to use the toString method instead. This results in unique hash values based on the entire method signature.

sbabcoc commented 2 years ago

Resolved by #123