search

sbt / sbt-jupiter-interface

Implementation of sbt's test interface for JUnit 5's Jupiter module
Apache License 2.0
33 stars 17 forks source link

Unable to Run Test Suite Class Not Enclosed #54

Closed D3Castro closed 2 months ago

D3Castro commented 2 years ago

I'm attempting to run test suites using sbt test, however, I am unable to resolve a runtime exception from the dispatcher. I've created the below repository to replicate the issue. https://github.com/D3Castro/Junit5ModuleTestSuite

[error] WARNING: TestExecutionListener [net.aichler.jupiter.internal.event.Dispatcher] threw exception for method: executionFinished(TestIdentifier [uniqueId = [engine:junit-platform-suite]/[suite:com.test.TestSuite]/[engine:junit-jupiter]/[class:com.test.junit.SimpleClassTest], parentId = [engine:junit-platform-suite]/[suite:com.test.TestSuite]/[engine:junit-jupiter], displayName = 'SimpleClassTest', legacyReportingName = 'com.test.junit.SimpleClassTest', source = ClassSource [className = 'com.test.junit.SimpleClassTest', filePosition = null], tags = [], type = CONTAINER], TestExecutionResult [status = SUCCESSFUL, throwable = null]) [error] java.lang.RuntimeException: Test class com.test.junit.SimpleClassTest is not enclosed by com.test.TestSuite [error] at net.aichler.jupiter.internal.event.TaskName.nestedSuiteId(TaskName.java:135) [error] at net.aichler.jupiter.internal.event.TaskName.of(TaskName.java:110) [error] at net.aichler.jupiter.internal.event.Dispatcher.lambda$executionFinished$2(Dispatcher.java:99) [error] at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705) [error] at net.aichler.jupiter.internal.event.Dispatcher.executionFinished(Dispatcher.java:95) [error] at org.junit.platform.launcher.core.CompositeTestExecutionListener.lambda$executionFinished$10(CompositeTestExecutionListener.java:69)

maichler commented 2 years ago

Hi,

thank you for reporting this issue.

I'm on vacation and can't check it out right now but maybe someone else has an idea what's going on in the meantime?

On Tue, Sep 20, 2022, 00:35 D3Castro @.***> wrote:

I'm attempting to run test suites using sbt test, however, I am unable to resolve a runtime exception from the dispatcher. I've created the below repository to replicate the issue. https://github.com/D3Castro/Junit5ModuleTestSuite

[error] WARNING: TestExecutionListener [net.aichler.jupiter.internal.event.Dispatcher] threw exception for method: executionFinished(TestIdentifier [uniqueId = [engine:junit-platform-suite]/[suite:com.test.TestSuite]/[engine:junit-jupiter]/[class:com.test.junit.SimpleClassTest], parentId = [engine:junit-platform-suite]/[suite:com.test.TestSuite]/[engine:junit-jupiter], displayName = 'SimpleClassTest', legacyReportingName = 'com.test.junit.SimpleClassTest', source = ClassSource [className = 'com.test.junit.SimpleClassTest', filePosition = null], tags = [], type = CONTAINER], TestExecutionResult [status = SUCCESSFUL, throwable = null]) [error] java.lang.RuntimeException: Test class com.test.junit.SimpleClassTest is not enclosed by com.test.TestSuite [error] at net.aichler.jupiter.internal.event.TaskName.nestedSuiteId(TaskName.java:135) [error] at net.aichler.jupiter.internal.event.TaskName.of(TaskName.java:110) [error] at net.aichler.jupiter.internal.event.Dispatcher.lambda$executionFinished$2(Dispatcher.java:99) [error] at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705) [error] at net.aichler.jupiter.internal.event.Dispatcher.executionFinished(Dispatcher.java:95) [error] at org.junit.platform.launcher.core.CompositeTestExecutionListener.lambda$executionFinished$10(CompositeTestExecutionListener.java:69)

— Reply to this email directly, view it on GitHub https://github.com/sbt/sbt-jupiter-interface/issues/54, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFBC4NIGTLUYSCGTHELHCLV7DTB5ANCNFSM6AAAAAAQQQTXYU . You are receiving this because you are subscribed to this thread.Message ID: @.***>

D3Castro commented 1 year ago

Is there anything related to the setup which may be causing this that I could investigate?

maichler commented 1 year ago

I think the issue is caused by using @Suite, which is part of the relatively new suite support in JUnit 5. Suite support comes with its own engine (engine:junit-platform-suite) which modifies the test identifier in such a way that TaskName can't handle it correctly anymore and makes the wrong assumptions.

To be sure I have to investigate a bit more and try to isolate it in a test case to see how we can fix it. But this might take a while I'm afraid.

Best bet for a short term solution would be to not use @Suite for now, if that's possible.

kamilkloch commented 1 year ago

Any chances for @Suite support? :)

Nezisi commented 1 year ago

The original problem is - like @maichler explained – that TaskName cannot handle the name. TaskName was added to handle nested tests (@Nested annotation) properly.

Credit for the idea behind the fix goes to @maichler, too.

As the JUnit 5 suite support is added via an own engine, we can just extend the check in TaskName for the nested tests to additionally check if the test engine is the JUnit5-Engine.

Just to be on the safe side, I added nested tests inside the suite definition and the test ran by the suite definition to make sure I didn't break it.

https://github.com/Nezisi/sbt-jupiter-interface/compare/bump-sbt-1_8...Nezisi:sbt-jupiter-interface:fix-junit5-suite-support

The PR is based up on the SBT 1.9 pull request, I'll wait with the PR until it's merged.

eed3si9n commented 2 months ago

Here's https://github.com/sbt/sbt-jupiter-interface/pull/87, which takes the above branch and applies it on top of main now that I've bumped sbt version etc.