helmethair-co / scalatest-junit-runner

JUnit 5 runner for Scalatest
MIT License
37 stars 10 forks source link

Tests do not run when using maven-surefire-plugin version 3.0.0-M7 #90

Closed anatolyra closed 1 year ago

anatolyra commented 1 year ago

When using maven-surefire-plugin version 3.0.0-M7 tests do not run from command-line.

Steps to reproduce the behavior:

  1. git clone scalatest-junit-runner
  2. Changed the version of maven-surefire-plugin to 3.0.0-M7
  3. Run in command-line: mvn clean test -Dtest=SomeCodeTest
  4. See output: Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

Expected behavior Two tests in 'SomeCodeTest' should run and succeed.

Build/Test environment:

anatolyra commented 1 year ago

Two more things that I have noticed:

The highest version which still works is 3.0.0-M3. When running with versions 3.0.0-M4 and 3.0.0-M5 I get an error:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test (default-test) on project scalatest-junit-runner-maven-example: No tests were executed!  (Set -DfailIfNoTests=false to ignore this error.) -> [Help 1]

But when running with versions 3.0.0-M6/7 I don't get any error:

CustomListener: execution FINISHED: SomeCodeTest with result: SUCCESSFUL
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.069 s - in co.helmethair.scalatest.example.SomeCodeTest
CustomListener: execution FINISHED: scalatest with result: SUCCESSFUL
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  4.620 s
[INFO] Finished at: 2022-09-22T21:56:15+03:00
[INFO] ------------------------------------------------------------------------
giurim commented 1 year ago

Thank you for the report and the patience!

The highest version which still works is 3.0.0-M3. When running with versions 3.0.0-M4 and 3.0.0-M5 I get an error:

The cause of this is that there is no really good way (or I did not figured out) to report tests which are not strictly functions:

executionFinished(TestIdentifier [uniqueId = '[engine:scalatest]/[suite:co.helmethair.scalatest.example.SomeCodeTest]/[test:someFunc in SomeCode calculates square]', parentId = '[engine:scalatest]/[suite:co.helmethair.scalatest.example.SomeCodeTest]', displayName = 'someFunc in SomeCode calculates square', legacyReportingName = 'someFunc in SomeCode calculates square', source = MethodSource [className = 'co.helmethair.scalatest.example.SomeCodeTest', methodName = 'someFunc in SomeCode calculates square', methodParameterTypes = null], tags = [], type = TEST], TestExecutionResult [status = SUCCESSFUL, throwable = null])

The important part here is the methodParameterTypes which is null because when create the TestSource object I call MethodSource.from(containingSuite.getSuiteId(), testName)); which sets the methodParameterTypes to null. Passing a third parameter (an empty string) seems to solved the problem for M4, M5, M6 and M7.

It is unfortunate that M6/M7 even swallowed the exception, thank you very much for the additional report it helped to track down the problem a lot.

Expect a minor version released with the fix soon.

anatolyra commented 1 year ago

@giurim thank you! I appreciate that.

giurim commented 1 year ago

0.1.12 released, should be available on Maven Central soon

anatolyra commented 1 year ago

Thank you! This solves the issue.