search

mockk / mockk

mocking library for Kotlin
https://mockk.io
Apache License 2.0
5.44k stars 346 forks source link

ConfirmVerified not isolated per tests #821

Open dekar91 opened 2 years ago

dekar91 commented 2 years ago

Hi! I am not sure if it is desired behavior, but it started failing since 1,2.4. Now, confirmVerified fails not only if there are no verified calls in the current test but also in the previous one.

Prerequisites

Please answer the following questions for yourself before submitting an issue.

Expected Behavior

Expected, confirmVerified checks only mock of the current scope.

Current Behavior

confirmVerified fails if there were unchecked calls in previous tests

Context

Failure Logs

Stack trace #### Stack trace ``` // -----------------------[ YOUR STACK STARTS HERE ] ----------------------- > Task :processResources NO-SOURCE > Task :processTestResources NO-SOURCE > Task :compileKotlin w: /home/dekar/projects/kotlin-flow/src/main/kotlin/Main.kt: (7, 10): Parameter 'args' is never used > Task :compileJava NO-SOURCE > Task :classes UP-TO-DATE > Task :compileTestKotlin > Task :compileTestJava NO-SOURCE > Task :testClasses UP-TO-DATE > Task :test OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended Verification acknowledgment failed Verified call count: 0 Recorded call count: 1 Not verified calls: 1) SupplierInterface(#1).supply() Stack traces: 1) io.mockk.impl.InternalPlatform.captureStackTrace (InternalPlatform.kt:125) io.mockk.impl.stub.MockKStub.handleInvocation (MockKStub.kt:250) io.mockk.impl.instantiation.JvmMockFactoryHelper$mockHandler$1.invocation (JvmMockFactoryHelper.kt:23) io.mockk.proxy.jvm.advice.Interceptor.call (Interceptor.kt:21) io.mockk.proxy.jvm.advice.BaseAdvice.handle (BaseAdvice.kt:42) io.mockk.proxy.jvm.advice.jvm.JvmMockKProxyInterceptor.interceptNoSuper (JvmMockKProxyInterceptor.java:45) SupplierInterface$Subclass0.supply (-:-1) Producer.produce (Producer.kt:4) Test1$1$1$1.invokeSuspend (Test1.kt:15) Test1$1$1$1.invoke (Test1.kt:-1) Test1$1$1$1.invoke (Test1.kt:-1) io.kotest.core.spec.style.scopes.DescribeSpecContainerScope$it$3.invokeSuspend (DescribeSpecContainerScope.kt:107) io.kotest.core.spec.style.scopes.DescribeSpecContainerScope$it$3.invoke (DescribeSpecContainerScope.kt:-1) io.kotest.core.spec.style.scopes.DescribeSpecContainerScope$it$3.invoke (DescribeSpecContainerScope.kt:-1) io.kotest.engine.test.TestCaseExecutor$execute$innerExecute$1.invokeSuspend (TestCaseExecutor.kt:76) io.kotest.engine.test.TestCaseExecutor$execute$innerExecute$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.TestCaseExecutor$execute$innerExecute$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.interceptors.CoroutineDebugProbeInterceptor.intercept (CoroutineDebugProbeInterceptor.kt:29) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invokeSuspend (TestCaseExecutor.kt:85) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.interceptors.InvocationTimeoutInterceptor$intercept$3.invokeSuspend (InvocationTimeoutInterceptor.kt:43) io.kotest.engine.test.interceptors.InvocationTimeoutInterceptor$intercept$3.invoke (InvocationTimeoutInterceptor.kt:-1) io.kotest.engine.test.interceptors.InvocationTimeoutInterceptor$intercept$3.invoke (InvocationTimeoutInterceptor.kt:-1) kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturnIgnoreTimeout (Undispatched.kt:100) kotlinx.coroutines.TimeoutKt.setupTimeout (Timeout.kt:146) kotlinx.coroutines.TimeoutKt.withTimeoutOrNull (Timeout.kt:103) io.kotest.engine.test.interceptors.InvocationTimeoutInterceptor.intercept (InvocationTimeoutInterceptor.kt:42) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invokeSuspend (TestCaseExecutor.kt:85) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.TestInvocationInterceptor$intercept$2$3.invokeSuspend (TestInvocationInterceptor.kt:36) io.kotest.engine.test.TestInvocationInterceptor$intercept$2$3.invoke (TestInvocationInterceptor.kt:-1) io.kotest.engine.test.TestInvocationInterceptor$intercept$2$3.invoke (TestInvocationInterceptor.kt:-1) io.kotest.mpp.ReplayKt.replay (replay.kt:18) io.kotest.engine.test.TestInvocationInterceptor$intercept$2.invokeSuspend (TestInvocationInterceptor.kt:31) io.kotest.engine.test.TestInvocationInterceptor$intercept$2.invoke (TestInvocationInterceptor.kt:-1) io.kotest.engine.test.TestInvocationInterceptor$intercept$2.invoke (TestInvocationInterceptor.kt:-1) kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn (Undispatched.kt:89) kotlinx.coroutines.CoroutineScopeKt.coroutineScope (CoroutineScope.kt:264) io.kotest.engine.test.TestInvocationInterceptor.intercept (TestInvocationInterceptor.kt:30) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invokeSuspend (TestCaseExecutor.kt:85) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.interceptors.TimeoutInterceptor.intercept (TimeoutInterceptor.kt:33) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invokeSuspend (TestCaseExecutor.kt:85) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.interceptors.BlockedThreadTimeoutInterceptor.intercept (BlockedThreadTimeoutInterceptor.kt:74) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invokeSuspend (TestCaseExecutor.kt:85) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.interceptors.CoroutineLoggingInterceptor.intercept (CoroutineLoggingInterceptor.kt:30) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invokeSuspend (TestCaseExecutor.kt:85) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.interceptors.SoftAssertInterceptor.intercept (SoftAssertInterceptor.kt:27) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invokeSuspend (TestCaseExecutor.kt:85) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.interceptors.AssertionModeInterceptor.intercept (AssertionModeInterceptor.kt:25) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invokeSuspend (TestCaseExecutor.kt:85) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.interceptors.LifecycleInterceptor.intercept (LifecycleInterceptor.kt:50) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invokeSuspend (TestCaseExecutor.kt:85) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.interceptors.EnabledCheckInterceptor.intercept (EnabledCheckInterceptor.kt:31) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invokeSuspend (TestCaseExecutor.kt:85) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.interceptors.TestCaseExtensionInterceptor$intercept$2.invokeSuspend (TestCaseExtensionInterceptor.kt:24) io.kotest.engine.test.interceptors.TestCaseExtensionInterceptor$intercept$2.invoke (TestCaseExtensionInterceptor.kt:-1) io.kotest.engine.test.interceptors.TestCaseExtensionInterceptor$intercept$2.invoke (TestCaseExtensionInterceptor.kt:-1) io.kotest.engine.test.TestExtensions.intercept (TestExtensions.kt:154) io.kotest.engine.test.interceptors.TestCaseExtensionInterceptor.intercept (TestCaseExtensionInterceptor.kt:24) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invokeSuspend (TestCaseExecutor.kt:85) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.interceptors.CoroutineErrorCollectorInterceptor$intercept$3.invokeSuspend (CoroutineErrorCollectorInterceptor.kt:28) io.kotest.engine.test.interceptors.CoroutineErrorCollectorInterceptor$intercept$3.invoke (CoroutineErrorCollectorInterceptor.kt:-1) ... Verified call count: 0 Recorded call count: 1 Not verified calls: 1) SupplierInterface(#1).supply() Stack traces: 1) io.mockk.impl.InternalPlatform.captureStackTrace (InternalPlatform.kt:125) io.mockk.impl.stub.MockKStub.handleInvocation (MockKStub.kt:250) io.mockk.impl.instantiation.JvmMockFactoryHelper$mockHandler$1.invocation (JvmMockFactoryHelper.kt:23) io.mockk.proxy.jvm.advice.Interceptor.call (Interceptor.kt:21) io.mockk.proxy.jvm.advice.BaseAdvice.handle (BaseAdvice.kt:42) io.mockk.proxy.jvm.advice.jvm.JvmMockKProxyInterceptor.interceptNoSuper (JvmMockKProxyInterceptor.java:45) SupplierInterface$Subclass0.supply (-:-1) Producer.produce (Producer.kt:4) Test1$1$1$1.invokeSuspend (Test1.kt:15) Test1$1$1$1.invoke (Test1.kt:-1) Test1$1$1$1.invoke (Test1.kt:-1) io.kotest.core.spec.style.scopes.DescribeSpecContainerScope$it$3.invokeSuspend (DescribeSpecContainerScope.kt:107) .. Not verified calls: 1) SupplierInterface(#1).supply() Stack traces: 1) io.mockk.impl.InternalPlatform.captureStackTrace (InternalPlatform.kt:125) io.mockk.impl.stub.MockKStub.handleInvocation (MockKStub.kt:250) io.mockk.impl.instantiation.JvmMockFactoryHelper$mockHandler$1.invocation (JvmMockFactoryHelper.kt:23) io.mockk.proxy.jvm.advice.Interceptor.call (Interceptor.kt:21) io.mockk.proxy.jvm.advice.BaseAdvice.handle (BaseAdvice.kt:42) io.mockk.proxy.jvm.advice.jvm.JvmMockKProxyInterceptor.interceptNoSuper (JvmMockKProxyInterceptor.java:45) SupplierInterface$Subclass0.supply (-:-1) Producer.produce (Producer.kt:4) Test1$1$1$1.invokeSuspend (Test1.kt:15) Test1$1$1$1.invoke (Test1.kt:-1) Test1$1$1$1.invoke (Test1.kt:-1) io.kotest.core.spec.style.scopes.DescribeSpecContainerScope$it$3.invokeSuspend (DescribeSpecContainerScope.kt:107) io.kotest.core.spec.style.scopes.DescribeSpecContainerScope$it$3.invoke (DescribeSpecContainerScope.kt:-1) io.kotest.core.spec.style.scopes.DescribeSpecContainerScope$it$3.invoke (DescribeSpecContainerScope.kt:-1) io.kotest.engine.test.TestCaseExecutor$execute$innerExecute$1.invokeSuspend (TestCaseExecutor.kt:76) io.kotest.engine.test.TestCaseExecutor$execute$innerExecute$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.TestCaseExecutor$execute$innerExecute$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.interceptors.CoroutineDebugProbeInterceptor.intercept (CoroutineDebugProbeInterceptor.kt:29) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invokeSuspend (TestCaseExecutor.kt:85) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.TestCaseExecutor$execute$2$1.invoke (TestCaseExecutor.kt:-1) io.kotest.engine.test.interceptors.InvocationTimeoutInterceptor$intercept$3.invokeSuspend (InvocationTimeoutInterceptor.kt:43) io.kotest.engine.test.interceptors.InvocationTimeoutInterceptor$intercept$3.invoke (InvocationTimeoutInterceptor.kt:-1) ... Test2 > Producer 1 > Test2.Test producer 1 FAILED java.lang.AssertionError at Test2.kt:15 Caused by: java.lang.AssertionError at Test2.kt:15 2 tests completed, 1 failed > Task :test FAILED FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':test'. > There were failing tests. See the report at: file:///home/dekar/projects/kotlin-flow/build/reports/tests/test/index.html * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. * Get more help at https://help.gradle.org BUILD FAILED in 4s 3 actionable tasks: 3 executed // -----------------------[ YOUR STACK TRACE ENDS HERE ] ----------------------- ```

Minimal reproducible code (the gist of this issue)

// -----------------------[ GRADLE DEFINITIONS ] -----------------------
dependencies {
    testImplementation(kotlin("test"))
    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.1")

    testImplementation ("io.kotest:kotest-runner-junit5-jvm:5.3.0")
    testImplementation ("io.mockk:mockk:1.12.4")
}
// -----------------------[ YOUR CODE STARTS HERE ] -----------------------

interface SupplierInterface {
    fun supply()
}

class Producer(private val supplier: SupplierInterface) {

    fun produce() {
        supplier.supply()
    }
}

// Test1.kt
class Test1:DescribeSpec({
    describe("Producer 1") {
        it("Test producer 1"){
            val supplier = mockk<SupplierInterface>()
            justRun { supplier.supply() }

            val producer = Producer(supplier)

            // then

            producer.produce()
//        !!!  No verify here

        }
    }
})

// Test2.kt
class Test2:DescribeSpec({
    describe("Producer 1") {
        it("Test producer 1"){
            val supplier = mockk<SupplierInterface>()
            justRun { supplier.supply() }

            val producer = Producer(supplier)

            // then
            producer.produce()
            verify { supplier.supply() }
            confirmVerified()
        }
    }
})

Run tests ./gradle test

Output:


Verification acknowledgment failed

Verified call count: 0
Recorded call count: 1

Not verified calls:
1) SupplierInterface(#1).supply()

....
dekar91 commented 2 years ago

Hello here. Any updates?

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. If you are sure that this issue is important and should not be marked as stale just ask to put an important label.

eirikvaa commented 1 year ago

Is there any update on this? We still see this issue in v1.13.5.

maximusgla commented 1 year ago

I had the same issue when using 1.13.5, it disappeared after upgrading mockk to 1.13.8.

luyen-metro commented 10 months ago

Any updates? I have the same issue when trying to upgrade mockk from 1.12.3 to 1.12.4. Also 1.13.8 does not work either.

u6f6o commented 8 months ago

I am having the same issue, applying: 

 @BeforeEach
    fun resetMockks() {
        clearMocks(
            eventPublisher,
            answers = false,
            recordedCalls = true,
            childMocks = false,
            verificationMarks = true,  // set this to true to have verifyAll be reset as well
            exclusionRules = false
        )
    }

helped in my case.

StenSoft commented 1 month ago

Still happens in 1.13.12 and JUnit4, even with

@get:Rule
val mockkRule = MockKRule(this)

which I would assume should reset verification marks in finished (but it does not).

A workaround is to add:

// Workaround for MockK bug #821 <https://github.com/mockk/mockk/issues/821>
@AfterTest
fun resetMocks() = clearAllMocks()