Contract verifiable mocks using shared examples (discussion)

[lukeredpath]

Background

Apologies if this isn't the best place to have this discussion. What follows is a largely incomplete idea but one I think is potentially useful and I wanted to get this out of my head and see firstly, if other people think this could be a good idea and secondly, if its technically feasible.

If people think this idea is workable, then I'm very happy to help with implementation as it's been a while since I contributed to RSpec (or indeed any open source project).

Some of the ideas that follow are inspired by the work of Sandi Metz in her Ruby book and also JB Rainsberger and the idea of "contract tests". Sorry if this is a bit long-winded.

Expected behavior

I'm currently working on some code in a Rails app that defines an abstract role. The role is implemented as a Ruby mixin.

It will typically be implemented by an ActiveRecord object but it doesn't have to be as there's no tight coupling.

The core behaviour is defined in a module. This module expects the object it extends to conform to a particular interface in order for it's behaviour to work.

I've tested this behaviour by writing shared examples for the mixin - both in terms of behaviour and the interface contract they require of the object being extended.

Elsewhere I'm representing this role with a mock but I have no way to verify that my mock conforms to the interface defined by the module.

Mixins aren't the only way of defining a role. We can also have more implicit roles, or "duck types". As Ruby doesn't have interfaces, we can instead use our tests to define the behaviour and contract of a duck type:

RSpec.shared_examples_for "duck" do
  describe 'contract' do
    it { is_expected.to respond_to(:quack) }
  end
end

Now we have a concrete object:

class Mallard
  def quack
  end
end

And we verify it's behaviour:

describe Mallard do
  it_behaves_like 'duck'
end

Success!

Now, SomethingProcessable doesn't live in a isolated world. It's used as a collaborator with something else. Here's a contrived example:

class DuckAlarm
  def initialize(duck)
    @duck = duck
  end

  def trigger
    duck.quack if some_condition_is_met?
  end
end

Our tests for this object are only concerned with testing that the alarm is triggered under the right conditions. We don't care what the duck is. We have a clearly defined role so we are happy to mock it (mock roles, not types):

let(:duck) { double('duck') }

subject { DuckAlarm.new(duck) }

it 'quacks the duck when some condition is met' do
  expect(duck).to receive(:quack)
  subject.trigger
end

So far, so good. Now we run into the familiar issue that is often used to criticise mocking, especially in the dynamic world of Ruby. What happens if our interface changes?

We decide that Duck.quack should actually be Duck.quack(volume).

We update our shared examples and Mallard. But we have a bug. DuckAlarm is still using the old interface but it's specs haven't failed because we haven't updated our mock.

Actual behavior

So what could we do about this? Clearly the fault doesn't lie with mocking as a concept - it is us who has forgotten to update our mock, which, being as much an implementation of our role as our concrete types should be updated, but we're missing the feedback to make this obvious.

RSpec already gives us a form of verifiable mock - we could use instance_double('Mallard'). But this feels like a code smell - we could have many ducks. Which one do we choose? Why do we care? Aren't we supposed to be mocking roles, not types?

We already have a definition of the Duck contract we could use. What if our mocks could be verified against this contract instead?

So, given our above contract shared examples, what if we could have:

# duck is auto-verified against duck.contract
let(:duck) { contract_verifiable_mock('duck') } 

Or if the auto-matching of names to contracts feels too much like magic, we could be more explicit:

let(:duck) do
  contract_verifiable_mock('duck', it_behaves_like: 'duck contract')
end

Is this a feasible idea? Could this be made to work? Am I completely crazy? Discuss!

[lukeredpath]

As a proof of concept, I'm keen to hack something together in my current project to verify my shared behaviour contract. In terms of structure, I'm working with the following.

Given a module and a class that mixes it in:

module MyBehaviour
end

class Something
  include MyBehaviour
end

And some shared examples:

shared_examples_for "my behaviour contract" do
  it { is_expected.to respond_to(:something) }
end

shared_examples_for "my behaviour" do
  include_examples "my behaviour contract"

  it "does something interesting" { }
end

I'm separating the contract specs so I can use them separately (I want my mocks to conform to the contract but not the behaviour as mocks don't implement behaviour as such) but also including them as a sanity check.

In my mind, I'm thinking a starting point would be being able to write a matcher that verified an object against a shared example group using self as the subject. Using internal API, is there any way I could implement something like:

expect(mock).to behave_like('my behaviour contract')

(also, in the name of readability I'm thinking :conform_to would make a good alias for :behaves_like for this particular use case).

[lukeredpath]

OK, here's where I'm at in my current codebase. It's not automatic, but it's a start. It depends on the behaviour above, where you have one shared example group for your contract and one for your behaviour (with the contract examples optionally included).

module DoubleContractVerification
  def verify_double(double_label, conforms_to:)
    context "verify double(:#{double_label})" do
      subject { __send__(double_label) }
      it_behaves_like conforms_to
    end
  end
end

RSpec.configure do |config|
  config.extend DoubleContractVerification
end

And in your example:

RSpec.shared_examples_for 'fooable contract' do
  it { is_expected.to respond_to(:foo) }
end

RSpec.shared_examples_for 'fooable' do
  include_examples 'fooable contract'
  it('does something interesting') { }
end

RSpec.describe "HTTPFooable" do
  it_behaves_like "fooable"
end

RSpec.describe "Uses Fooable" do
  let(:fooable) { double('fooable', fooable: 'just stubbing') }
  verify_double :fooable, conform_to: 'fooable contract'
end

There is a major limitation of this approach and that is whilst it's easy to define any query stubs on a double at the point you create your double, typically doubles do not respond to commands until we set expectations on them. If we set expectations in an individual test, then the double might not yet conform to the contract at definition time.

I'm not sure what the best approach to solving this is. Perhaps the answer is that you should allow commands at definition time, but change these to expectations when you need to verify that it has been called? This might make sense for most use cases anyway - if you're following the "one expectation per test" rule, you might have multiple tests where a command message is sent to your mock, but only in one of these are you interested in verifying this. You would still need to use allow in your other tests so you don't get an unexpected message error. A common refactoring might be to move this setup into a before/let block so perhaps it aligns well.

[JonRowe]

I feel like you could already achieve this using something like this:

RSpec.shared_examples_for("ducks") do
  let(:contract) do
    Class.new do
      def quack(text)
        # ...
      end
    end
  end
  let(:duck) { instance_double(contract) }
end

I think this is an interesting idea though, having the ability to define a definitive contract and use a double that matches that contract, I feel the verifying double stuff could do this already if we gave it a class like this to make it work?

[myronmarston]

@lukeredpath there are a couple existing projects that might do what you want (or something similar to what you want):

• https://github.com/cvincent/rspec-fire-roles (targeted rspec-fire, which became rspec's verifying doubles when merged into rspec-mocks)
• https://github.com/psyho/bogus (particularly the contract testing feature)

[alexevanczuk]

@lukeredpath

Where I work we created a custom matcher using the contracts gem. I talk about it more in this issue. We are working on open-sourcing it but not quite there. With this matcher, we verify that when we set the expectation against the target (the duck), we verify that the parameters match the specified target. I think you could do the same thing here. Create a custom matcher which verifies that the method you are expecting is being received with the correct parameter types.

I don't think that instance_double is a Code smell, and I suppose it depends on how you use it. If you have a method that takes in a lot of Ducks as you mentioned, then having a TestDuck type defined in your test file (or test helpers somewhere) and using the approach @JonRowe mentioned could give you a contract verifying double without needing to rewrite a lot of the double verification code that rspec has out of the box.

[kurko]

@lukeredpath I have been in your shoes since 2012, precisely because of JB Rainsberger (and later Sandi Metz). What I do is call it_behaves_like 'foo contract in every class that uses foo, so when the interface changes all dependent spec files will also fail.

Note: contracts with respond_to? won't cover argument changes.