Expectations should be verified when an exception is thrown from within a play block

[GoogleCodeExporter]

If an exception is thrown from a play block then the expectations are not 
asserted.  

This is particularly troublesome when using GMock with testing frameworks like 
Spock to check an exception has been thrown.

Example:

The following spock test should fail becuase the toString() method is not 
invoked.  But because the expectations are verified at the end of the play 
block, the test passes!

given:
   Integer blah = mock(Integer)
   blah.toString().returns('69')

when:
   play {
      throw new RuntimeException()
   }

then:
   thrown(RuntimeException)

A workaround is to use the following method as a replacement for play, which 
catches and rethrows exceptions after the expectations have been verified.

def simulate(Closure closure) {

    Exception exception = null
    def result = null

    play {
        try {
           result = closure.call()
        } catch (Exception ex) {
           exception = ex
        }
    }

    if (exception) {
        throw exception
    }

    return result
}

Original issue reported on code.google.com by russell....@gmail.com on 10 Jun 2011 at 5:36

[GoogleCodeExporter]

That is what we designed. However, now I think wrapping the exception and 
attaching the calling status(what has been called and what has not been called) 
is a good idea.

Original comment by JohnnyJianHY on 11 Jun 2011 at 1:51

• Changed state: Accepted
• Added labels: Milestone-0.9, Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

If we wrap the exception with our own type, say ExceptionWrapper, then you 
cannot assert throwing the wrapped exception. So the following test will fail:

shouldFail(MyException) {
  play {
    throw new MyException()
  }
}

Now I see what problem you have. It is easy to do it with shouldFail(), but for 
Spock, does the following work?

play {
  when:
    throw new RuntimeException()

  then:
    thrown(RuntimeException)
}

Obviously, it is not a good solution even it works. To make gmock work well 
with spock, a spock extension is needed, I think.

Original comment by JohnnyJianHY on 11 Jun 2011 at 3:34

• Changed state: New

[GoogleCodeExporter]

Original comment by JohnnyJianHY on 11 Jun 2011 at 3:36

• Removed labels: Milestone-0.9

[GoogleCodeExporter]

You said this is what we designed but it seems wrong. The problem really is 
that when an exception is thrown we don't even try to verify the mocks. 

I think we should catch any exception. At the end of the play block verify all 
mocks as we normally do and then rethrow any exception that occured.

Original comment by julien.g...@gmail.com on 11 Jun 2011 at 4:09

• Changed state: Accepted

[GoogleCodeExporter]

Yes, when an exception is thrown we should not verify the mocks.

First, if the exception is unexpected, it must be a bug and the user should fix 
it first. If we catch the exception and verify the mocks, the exception will be 
swallowed and the user will never be notified that the exception is thrown, 
because if the exception is unexpected, the expectation is usually mismatched, 
so an expectation mismatching exception will be thrown instead.

Like I said, it is a good idea to wrap the exception and attach the call 
status. But it doesn't work well with shouldFail().

Second, if the exception is expected, than the user should write the test like:

play {
  shouldFail(MyException) {
    throw new MyException()
  }
}

So the exception will be asserted and the mocks will be verified at the end of 
the play block.

The problem of using gmock in spock is that the exception assertion(the "then:" 
label) is outside the play block. So I think the best solution is to intercept 
the "when:" and "then:" labels and replay before "when:" and verify after 
"then:".

Original comment by JohnnyJianHY on 12 Jun 2011 at 3:00