Contract test failures are not clear

[ghostsquad]

The test output is not "clear" that a contract test failed, by looking only at the final output of rspec:

3 examples, 0 failures

https://www.relishapp.com/bogus/bogus/docs/contract-tests/contract-tests-with-mocks#fails-when-mocked-methods-are-not-called-on-real-object

I get an exit code 1 when I run rspec, and the output says that contracts aren't fulfilled, but it doesn't actually says that there are any failed examples.

$ rspec; echo "exitcode: $?"                                                                                                                                                                                                                               ‹ruby-2.3.0›
...
An error occurred in an `after(:suite)` hook.
Failure/Error: raise Bogus::ContractNotFulfilled.new(fake_name, missed: missed, actual: actual)

Bogus::ContractNotFulfilled:
  Contract not fullfilled for library!

  Missed interactions:
    - #has_book?("Moby Dick") => true

  Actual interactions:
    - #return("Moby Dick") => []
    - #checkout("Moby Dick") => "Moby Dick"
    - #has_book?("Moby Dick") => false
# /Users/wes/.rvm/gems/ruby-2.3.0/gems/bogus-0.1.6/lib/bogus/contracts/verifies_contracts.rb:14:in `verify'
...

3 examples, 0 failures

exitcode: 1

If I wasn't paying attention to the exit code, I would not have noticed this failed test.

[psyho]

Thank you for your feedback.

It's kind of tricky to make contract tests count as examples for RSpec. The reason for that is that contract tests need to be checked at the very end of the spec run - after all the other examples have run. As far as I know, there is no other way to make that happen in RSpec than to put the execution in an after(:suite) callback.

If you have an idea about how that could be accomplished, I would be really glad to hear it.

[ghostsquad]

I also noticed that if the tests are split up (like with circleci parallelism), the contract tests don't work. This is a harder problem to solve, because the validation requirements are built up through the use of rspec expect assertions.