spekframework / spek

A specification framework for Kotlin
Other
2.23k stars 179 forks source link

Spek tests are executed along regular JUnit 5 tests #916

Open R4md4c opened 4 years ago

R4md4c commented 4 years ago

If I try to run a single regular JUnit5 test from the IDE (Android Studio 4.0.1), I find that all the Spek tests are executed along the regular one. I've included a test project as a zip file.

Just perform a run from IDE on the CaculatorJUnitTest and you should see something similar to the screenshot below.

SpekIssueReproducer.zip

image

R4md4c commented 4 years ago

Also when running tests from gradle I notice that Spek and JUnit5 are both running, even-though I specified the --tests to run just the JUnit one.

$ ./gradlew testDebug --tests "com.example.spekissuereproducer.CaculatorJUnitTest" --rerun-tasks

> Task :app:testDebugUnitTest

CaculatorSpekTest > add should return 5 PASSED

CaculatorJUnitTest > add() PASSED

BUILD SUCCESSFUL in 2s
13 actionable tasks: 13 executed

EDIT: I had to add junit-jupiter to the list of engines included to make gradlew test detect the JUnit test class.

molszewski commented 4 years ago

We're experiencing the same issue - when we run any test via IntelliJ test runner or ./gradlew test --tests command line Spek2 tests are running as well.

Our build.gradle contains the following setup:

test {
    useJUnitPlatform() {
        includeEngines 'spek2'
        includeEngines 'junit-jupiter'
    }

Would appreciate if you could advise if more setup is required or a workaround can be applied to address this issue. Thank you.

R4md4c commented 4 years ago

Unfortunately I'm still suffering from this problem, one workaround that I discovered is executing single test methods doesn't trigger this bug, unlike when trying to run the entire class. It's not ideal but it's less time consuming than trying running an entire class and executing other Spek tests with it.

tikal commented 3 years ago

I believe this issue comes from Spek not applying the tests filter. That's an issue I experienced as well, right now it is not possible to only run some specific Spek test(s) via Junit runner...