Mockito should reset ThreadLocal variables which hold test progress before @Before methods

[GoogleCodeExporter]

What steps will reproduce the problem?
1.Force JUnit to reuse same thread for consecutive test exectuion
2.You will get  
org.mockito.exceptions.base.MockitoException: 
Cannot stub a void method with a return value! 
error

following code snippet should be used to reset ThreadLocal state before
running each test method:

new ThreadSafeMockingProgress().reset();
(org.mockito.internal.progress.ThreadSafeMockingProgress)

Otherwise there could be unpredictable errors which starts to appear after
a number of tests using mockito / forced to the same thread.

Original issue reported on code.google.com by izapol...@gmail.com on 20 Mar 2009 at 2:38

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Following code could be used for workaround:

package utilities.test.mockito;

import org.junit.internal.runners.InitializationError;
import org.junit.internal.runners.MethodRoadie;
import org.junit.internal.runners.TestMethod;
import org.junit.runner.Description;
import org.junit.runner.notification.RunNotifier;
import org.mockito.internal.progress.ThreadSafeMockingProgress;
import org.mockito.runners.MockitoJUnit44Runner;

import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;

public class MockitoRunner extends MockitoJUnit44Runner
{
    public MockitoRunner(Class<?> klass) throws InitializationError {
        super(klass);
    }

    @Override
    protected void invokeTestMethod(Method method, RunNotifier notifier) {
        Description description = methodDescription(method);
        Object test;
        try {
            test = createTest();
        } catch (InvocationTargetException e) {
            notifier.testAborted(description, e.getCause());
            return;
        } catch (Exception e) {
            notifier.testAborted(description, e);
            return;
        }
        TestMethod testMethod = wrapMethod(method);
        new MockitoStateResetterRoadie(test, testMethod, notifier, description)
                .run();
    }

    class MockitoStateResetterRoadie extends MethodRoadie
    {
        MockitoStateResetterRoadie(Object test, TestMethod method, RunNotifier
notifier, Description description) {
            super(test, method, notifier, description);
        }

        @Override
        public void runBeforesThenTestThenAfters(Runnable test) {
            new ThreadSafeMockingProgress().reset();
            super.runBeforesThenTestThenAfters(test);
        }
    }
}

Original comment by izapol...@gmail.com on 20 Mar 2009 at 2:52

[GoogleCodeExporter]

Hi,

1. Does it occur when you use @RunWith or the issue occurs also when run 
without a
runner?

2.

>1.Force JUnit to reuse same thread for consecutive test exectuion

How do I do it?

Original comment by szcze...@gmail.com on 27 Mar 2009 at 12:39

[GoogleCodeExporter]

1. Yes, it doesn't matter if it was using @RunWith with MockitoJUnit44Runner or 
just
basic JUnit runner

2. I will update on this question after short investigation.

Original comment by izapol...@gmail.com on 27 Mar 2009 at 4:50

[GoogleCodeExporter]

I don't know how to reproduce this bug. How do you force jUnit to use same 
thread? Do
you experience problems when you run it from IDE or ant / maven?

Original comment by szcze...@gmail.com on 21 Apr 2009 at 10:19

[GoogleCodeExporter]

this happens to me when running with buildr when the following applies:

I use something like this in one test:

assertThat(null,org.mockito.Matchers.isNull());

assuming that this is just a regular hamcrest matcher I can use outside of 
Mockito.

this leaves mockingProgress with a non-empty matcher stack.  if the next test 
is run
in the same thread, when it tries to Mockito.mock(Object.class) then the state 
is
invalid --> ka-boom.

I use my own @RunWith annotation.

Original comment by miaubiz@gmail.com on 22 Apr 2009 at 12:48

[GoogleCodeExporter]

This is a simple test case that will demonstrate that by default JUnit 4.4 uses
single-thread execution model. Please note that if there is a timeout in a test
method, then it will use new thread for execution of each test method:

@RunWith(org.mockito.runners.MockitoJUnit44Runner.class)
public class ThreadReuseTest
{
    public static ThreadLocal<Class> l = new ThreadLocal<Class>();

    @Test (timeout = 100)
    public void test1 () throws Exception {
        assertNull (l.get());
        l.set(getClass());
    }

    @Test
    public void test2 () throws Exception {
        assertNull (l.get());
        l.set(getClass());
    }

    @Test (timeout = 100)
    public void test3 () throws Exception {
        assertNull (l.get());
        l.set(getClass());
    }

    @Test (timeout = 100)
    public void test4 () throws Exception {
        assertNull (l.get());
        l.set(getClass());
    }

    @Test
    public void test5 () throws Exception {
        assertNull (l.get());
        l.set(getClass());
    }

}

Original comment by izapol...@gmail.com on 22 Apr 2009 at 2:26

[GoogleCodeExporter]

miaubiz,

Mockito API doesn't provide any specific implementations of hamcrest Matchers 
(apart
from abstract ArgumentMatcher). org.mockito.Matchers.isNull() doesn't return a
Matcher but null therefore you cannot do this:

assertThat(null,org.mockito.Matchers.isNull());

>this leaves mockingProgress with a non-empty matcher stack.  if the next test 
is run
>in the same thread, when it tries to Mockito.mock(Object.class) then the state 
is
>invalid --> ka-boom.

True. Trunk version of Mockito has better defect localization - you will have a
navigable stack trace element where argument matcher was misused.

Original comment by szcze...@gmail.com on 22 Apr 2009 at 9:05

[GoogleCodeExporter]

izapolsky,

I'd rather get to the bottom of the problem. I see resetting the state helped 
you out
but if we follow this idea then we'll lose framework validation. Example:

test1:
  //someone didn't use the framework correctly:
  verify(mock);

test2
  //if the state was reset then the user will never know test1 is incorrect

Can we somehow nail down the problem? Can you provide an example with Mockito 
failing?

Original comment by szcze...@gmail.com on 22 Apr 2009 at 9:22

[GoogleCodeExporter]

Cannot reproduce the problem

Original comment by szcze...@gmail.com on 17 Jun 2009 at 12:27

• Changed state: WontFix

[GoogleCodeExporter]

I suspect that i have this issue. When i run the test on my local machine its 
all good. 
But when the build server runs it (team-city) the tests asserts fail. :/ I have 
some 
test fixture setup in the @Before method like when and mock code.  Strange 
problem this 
is - yoda

Original comment by RikardQvarforth@gmail.com on 30 Nov 2009 at 7:32

[GoogleCodeExporter]

Can you confirm something for me:

1. When you run it from command line locally it passes?
2. When you run it from command line on CI box it passes?

Thanks!

Original comment by szcze...@gmail.com on 30 Nov 2009 at 1:46

• Changed state: New

[GoogleCodeExporter]

When i run all the tests in the test class then two tests fails on the assert. 
But when 
i run one singel faild test then it passes. One workaround this for me was to 
put the 
new ThreadSafeMockingProgress().reset(); in each failing test. Then the hole 
test class 
passes. 

Must be something with shared data between test's 

Original comment by RikardQvarforth@gmail.com on 30 Nov 2009 at 2:00

[GoogleCodeExporter]

Can you try one thing for me? Can you put Mockito.validateMockitoUsage() in 
@Before
the test case that is failing? Does it tell anything more?

Original comment by szcze...@gmail.com on 30 Nov 2009 at 2:05

[GoogleCodeExporter]

Sorry but im running mockito version 1.6 
So i dont have that method :/ validateMockitoUsage()

Original comment by RikardQvarforth@gmail.com on 2 Dec 2009 at 7:19

[GoogleCodeExporter]

I see. Can you upgrade :) ?

Original comment by szcze...@gmail.com on 2 Dec 2009 at 11:37

[GoogleCodeExporter]

Original comment by szcze...@gmail.com on 14 Feb 2010 at 10:10

• Changed state: WontFix