mockito / mockito-kotlin

Using Mockito with Kotlin
MIT License
3.11k stars 202 forks source link

Problem verifying method calls on spies #369

Closed MFAshby closed 4 years ago

MFAshby commented 4 years ago

I have a test class with 2 tests. The tests are trying to verify the behaviour of the subject by spying on it.

Expected behaviour: tests pass because the expected method call is made.

Actual behaviour: The first test passes, and the second (identical in this sample) test which attempts to verify receives an UnfinishedVerificationException

Sample code:

package com.mypackage

import com.nhaarman.mockitokotlin2.spy
import com.nhaarman.mockitokotlin2.times
import com.nhaarman.mockitokotlin2.verify
import org.junit.Test

open class SomeClass {
    fun doThing1() {
        doThing2()
    }

    fun doThing2() {
        println("Hey")
    }
}

class SpyProblem {
    private val underTest: SomeClass = spy(SomeClass())

    @Test
    fun `Can't verify spies`() {

        // WHEN
        underTest.doThing1()
        // THEN
        verify(underTest, times(1)).doThing2()
    }
    @Test
    fun `Can't verify spies 2`() {
        // WHEN
        underTest.doThing1()
        // THEN
        verify(underTest, times(1)).doThing2()
    }
}

Received logging:

/usr/lib/jvm/java-8-openjdk-amd64/bin/java -ea -Djdk.net.URLClassPath.disableClassPathURLCheck=true -Didea.test.cyclic.buffer.size=1048576 -javaagent:/snap/intellij-idea-ultimate/188/lib/idea_rt.jar=34129:/snap/intellij-idea-ultimate/188/bin -Dfile.encoding=UTF-8 -classpath /snap/intellij-idea-ultimate/188/lib/idea_rt.jar:/snap/intellij-idea-ultimate/188/plugins/junit/lib/junit5-rt.jar:/snap/intellij-idea-ultimate/188/plugins/junit/lib/junit-rt.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/charsets.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/cldrdata.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/dnsns.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/icedtea-sound.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/jaccess.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/java-atk-wrapper.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/localedata.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/nashorn.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/sunec.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/sunjce_provider.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/sunpkcs11.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/zipfs.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jce.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/jsse.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/management-agent.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/resources.jar:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar:/home/martin/pkb/kms/kms-client/kms-client-core/target/test-classes:/home/martin/pkb/kms/kms-client/kms-client-core/target/classes:/home/martin/.m2/repository/com/pkb/pkbcommon/infrastructure/4/infrastructure-4.jar:/home/martin/.m2/repository/javax/servlet/javax.servlet-api/4.0.1/javax.servlet-api-4.0.1.jar:/home/martin/.m2/repository/com/pkb/pkbcommon/date-time/4/date-time-4.jar:/home/martin/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar:/home/martin/.m2/repository/com/pkb/pkbcommon/testlogging/4/testlogging-4.jar:/home/martin/.m2/repository/org/apache/commons/commons-lang3/3.9/commons-lang3-3.9.jar:/home/martin/.m2/repository/org/jetbrains/annotations/16.0.3/annotations-16.0.3.jar:/home/martin/.m2/repository/io/vavr/vavr/0.10.0/vavr-0.10.0.jar:/home/martin/.m2/repository/io/vavr/vavr-match/0.10.0/vavr-match-0.10.0.jar:/home/martin/.m2/repository/com/pkb/crypto/commons-crypto/0.0.0-SNAPSHOT/commons-crypto-0.0.0-SNAPSHOT.jar:/home/martin/.m2/repository/org/immutables/value/2.7.5/value-2.7.5.jar:/home/martin/.m2/repository/com/google/guava/guava/27.0-jre/guava-27.0-jre.jar:/home/martin/.m2/repository/com/google/guava/failureaccess/1.0/failureaccess-1.0.jar:/home/martin/.m2/repository/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/home/martin/.m2/repository/com/google/code/findbugs/jsr305/3.0.2/jsr305-3.0.2.jar:/home/martin/.m2/repository/org/checkerframework/checker-qual/2.5.2/checker-qual-2.5.2.jar:/home/martin/.m2/repository/com/google/errorprone/error_prone_annotations/2.2.0/error_prone_annotations-2.2.0.jar:/home/martin/.m2/repository/com/google/j2objc/j2objc-annotations/1.1/j2objc-annotations-1.1.jar:/home/martin/.m2/repository/org/codehaus/mojo/animal-sniffer-annotations/1.17/animal-sniffer-annotations-1.17.jar:/home/martin/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.10.0/jackson-annotations-2.10.0.jar:/home/martin/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-guava/2.10.0/jackson-datatype-guava-2.10.0.jar:/home/martin/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.10.0/jackson-core-2.10.0.jar:/home/martin/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.10.0/jackson-databind-2.10.0.jar:/home/martin/.m2/repository/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar:/home/martin/.m2/repository/io/micrometer/micrometer-core/1.3.0/micrometer-core-1.3.0.jar:/home/martin/.m2/repository/org/hdrhistogram/HdrHistogram/2.1.11/HdrHistogram-2.1.11.jar:/home/martin/.m2/repository/org/latencyutils/LatencyUtils/2.0.3/LatencyUtils-2.0.3.jar:/home/martin/.m2/repository/junit/junit/4.12/junit-4.12.jar:/home/martin/.m2/repository/org/hamcrest/hamcrest-core/2.1/hamcrest-core-2.1.jar:/home/martin/.m2/repository/org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar:/home/martin/.m2/repository/com/github/karsaig/approvalcrest/0.21/approvalcrest-0.21.jar:/home/martin/.m2/repository/org/skyscreamer/jsonassert/1.5.0/jsonassert-1.5.0.jar:/home/martin/.m2/repository/com/vaadin/external/google/android-json/0.0.20131108.vaadin1/android-json-0.0.20131108.vaadin1.jar:/home/martin/.m2/repository/com/google/code/gson/gson/2.8.6/gson-2.8.6.jar:/home/martin/.m2/repository/com/nhaarman/mockitokotlin2/mockito-kotlin/2.2.0/mockito-kotlin-2.2.0.jar:/home/martin/.m2/repository/org/mockito/mockito-core/3.1.0/mockito-core-3.1.0.jar:/home/martin/.m2/repository/net/bytebuddy/byte-buddy/1.10.1/byte-buddy-1.10.1.jar:/home/martin/.m2/repository/net/bytebuddy/byte-buddy-agent/1.10.1/byte-buddy-agent-1.10.1.jar:/home/martin/.m2/repository/org/objenesis/objenesis/2.6/objenesis-2.6.jar:/home/martin/.m2/repository/org/jetbrains/kotlin/kotlin-stdlib/1.3.61/kotlin-stdlib-1.3.61.jar:/home/martin/.m2/repository/org/jetbrains/kotlin/kotlin-stdlib-common/1.3.61/kotlin-stdlib-common-1.3.61.jar com.intellij.rt.junit.JUnitStarter -ideVersion5 -junit4 com.mypackage.SpyProblem
Hey
Hey

org.mockito.exceptions.misusing.UnfinishedVerificationException: 
Missing method call for verify(mock) here:
-> at com.nhaarman.mockitokotlin2.VerificationKt.verify(Verification.kt:72)

Example of correct verification:
    verify(mock).doSomething()

Also, this error might show up because you verify either of: final/private/equals()/hashCode() methods.
Those methods *cannot* be stubbed/verified.
Mocking methods declared on non-public parent classes is not supported.

    at com.nhaarman.mockitokotlin2.SpyingKt.spy(Spying.kt:52)
    at com.mypackage.SpyProblem.<init>(SpyProblem.kt:19)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.junit.runners.BlockJUnit4ClassRunner.createTest(BlockJUnit4ClassRunner.java:217)
    at org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:266)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:263)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
    at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
    at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
    at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)

Process finished with exit code 255

mockito-kotlin version 2.2.0, kotlin version 1.3.61

MFAshby commented 4 years ago

Because the first test passes and not the second I assumed some state must be kept between the tests, so I modified it to reset the spy every time, however I still get the same issue!

    private lateinit var underTest: SomeClass 

    @Before
    fun setup() {
        underTest = spy(SomeClass())
    }
MFAshby commented 4 years ago

The same test written in java with regular Mockito works as expected:

package com.mypackage;

import static org.mockito.Mockito.spy;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;

import org.junit.Test;

class SomeClass {
    public void doThing1() {
        doThing2();
    }

    public void doThing2() {
        System.out.println("Hey");
    }
}

public class SpyingProblem {
    private SomeClass underTest = spy(new SomeClass());

    @Test
    public void cantVerifySpies() {

        // WHEN
        underTest.doThing1();

        // THEN
        verify(underTest, times(1)).doThing2();
    }

    @Test
    public void cantVerifySpies2() {

        // WHEN
        underTest.doThing1();

        // THEN
        verify(underTest, times(1)).doThing2();
    }
}
MFAshby commented 4 years ago

Interesting I think this might be something in mockito and not related to this wrapper. If I change the 'production' code to kotlin, and keep the test as java using mockito I get the same error:

SomeClass.kt

package com.pkb.kms.client.core

open class SomeClass {
    fun doThing1() {
        doThing2()
    }

    fun doThing2() {
        println("Hey")
    }
}

SpyingProblem.java:

package com.pkb.kms.client.core;

import static org.mockito.Mockito.spy;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;

import org.junit.Test;

public class SpyingProblem {
    private SomeClass underTest = spy(new SomeClass());

    @Test
    public void cantVerifySpies() {

        // WHEN
        underTest.doThing1();

        // THEN
        verify(underTest, times(1)).doThing2();
    }

    @Test
    public void cantVerifySpies2() {

        // WHEN
        underTest.doThing1();

        // THEN
        verify(underTest, times(1)).doThing2();
    }
}

Error logging:

org.mockito.exceptions.misusing.UnfinishedVerificationException: 
Missing method call for verify(mock) here:
-> at com.pkb.kms.client.core.SpyingProblem.cantVerifySpies(SpyingProblem.java:20)

Example of correct verification:
    verify(mock).doSomething()

I'll raise an issue with mockito instead!

bohsen commented 4 years ago

@MFAshby This is because your functions aren't open. They will be in java. Add open to both functions and it'll work. Otherwise use mockito-inline to enable this on final methods

MFAshby commented 4 years ago

@bohsen thanks, this worked!

What is confusing me though: why is it working without open for a single test? This led me to believe there was some state that was being retained between tests.

bohsen commented 4 years ago

What is confusing me though: why is it working without open for a single test? This led me to believe there was some state that was being retained between tests.

Can't say. Need an MWE to look at for further investigation if needed. Regarding shared state between unit tests. This is highly unlikely the reason as both JUnit 4 and 5 by default create a new instance of the test class before running each test method.

MFAshby commented 4 years ago

Minimal working exmple at https://github.com/MFAshby/spying-problem-sample @bohsen if you are in a position to investigate and have some time

bohsen commented 4 years ago

Might take a look at it, when I have time.

Thx for the sample and merry Christmas ☃️🎄

MFAshby commented 4 years ago

@bohsen thanks, have a good one!