Targeted Verification Support for Interaction Testing

[matttproud]

Requested feature

I am not sure if GoMock supports this natively or not.  If it does, it would be nice if the documentation reflected how to do interaction testing more clearly.  Interaction testing entails validating how a function is called, wherein a test should fail if a function isn’t called the right way.

The idea is that independent of the recorded expectations that one has the ability to verify that certain calls were made or not made, maybe ignoring the how other mocks associated with the controller are used.

This is partially expressable in GoMock today with the use of (*gomock.Controller).Finish, but it covers all mocks attached to that controller, meaning target verification is not possible.

This is a standard capability of other mocking suites.  If we take a look at Mockito, we can achieve targeted interaction testing with terminal statements as follows:

Database mockDb = mock(Database.class);  // create mock
Authorizer mockAuth = mock(Authorizer.class); // create mock

sut.operate(mockDb, mockAuth);  // use mock

verify(mockDb).PurgeEntries(); // only care that precisely this method is called
                               // no desire to verify mockAuth

You can see a rough sketch of this here: https://www.baeldung.com/mockito-verify.

I do not have a strong opinion on what the API should look like. Perhaps:

ctrl := gomock.NewController(t)

mockDB := mockdatabase.NewMockDatabase(ctrl) // verify only this one
mockAuth := mockauthorizer.NewMockAuthorizer(ctrl)

sut.Operate(mockDB, mockAuth)

ctrl.Finish()

mockDB.VERIFY().PurgeEntries()  

Why the feature is needed

Targeted interaction testing is bread and butter feature of mocking support libraries since time memorial. I appreciate that (*gomock.Controller).Finish can be used for validating ALL mocks within its scope, but that is unfortunately means I still need to record behavior (beyond mere classic stub returns) for all mock-based test doubles. That can get very fragile quickly, which is where the targeted verification comes in.

(Optional) Proposed solution A clear description of a proposed method for adding this feature to gomock.

I presume gomock.callSet would gain a new field for observed calls that basically appends all observed calls thereto:

type observedCall struct {
  Key callSetKey
  Call *Call
}

// callSet represents a set of expected calls, indexed by receiver and method
// name.
type callSet struct {
    // Calls that are still expected.
    expected map[callSetKey][]*Call
    // Calls that have been exhausted.
    exhausted map[callSetKey][]*Call

    // NEW

    // Calls that have been exhausted.
    observed []*observedCall
}

The Verify API would then consult observed and could perform presence, ordering, parameter, etc. validation.

Thank you for your consideration! I am happy to review code or design. I want to see this happen.

[codyoss]

Hey @matttproud,

Thank you for your feature request. To me this sounds a lot like #7 but a different API for the same result, correct me if I am wrong. This sort of thing can sort of be done today, but you would need to explicitly have a AnyTimes() expected call for each method that you don't want to "verify". I like personally like this explicitness but I understand the point that you may want to save a few extra lines of code in some cases.

If we did go with an approach like this is there a reason for VERIFY() to operate outside the scope of Finish()?

[matttproud]

Thank you for your feature request. To me this sounds a lot like #7 but a different API for the same result, correct me if I am wrong.

On a facial look, it does appear similar to #7. I am not too tied toward any API prescription so long as whatever is used is clear.

This sort of thing can sort of be done today, but you would need to explicitly have a AnyTimes() expected call for each method that you don't want to "verify". I like personally like this explicitness but I understand the point that you may want to save a few extra lines of code in some cases.

I do like explicitness as well personally, but this is not so much for me but rather to present GoMock itself in a technical documentation (cookbook) context at-parity with several other mocking libraries. Roughly what I think I would expect is that for each call to an unverified mock that such a mock would instead return the zero value responses for its given return values (a classical, dumb stub).

If we did go with an approach like this is there a reason for VERIFY() to operate outside the scope of Finish()?

That is a good question. I suspect that unfortunately though a mock-level attribute of {strict | loose} may still be too broad. Imagine I have a mock that has been given a set of recorded call and responses, but I really only care that a single thing has occurred for verification, leaving the rest to basically a "stub" like performance:

// Pardon the bad snippet.  It's late here, and I'm tired.

type UnderTest struct{}

func (sut UnderTest) Operate(db Database, auth Authorizer) {
  if !auth.IsSuperUser() {
    return
  }

  db.PurgeEntries()

  if auth.HasAttribute(authorization.MayDance) {
    dancelibrary.DoAJigOn(db)  // I do not want to care about this.
  }
}

func Test(t *testing.T) {
  ctrl := gomock.NewController(t)

  mockDB := mockdatabase.NewMockDatabase(ctrl) // verify only this one
  mockAuth := mockauthorizer.NewMockAuthorizer(ctrl)

  sut.Operate(mockDB, mockAuth)

  ctrl.Finish()

  mockDB.VERIFY().PurgeEntries()  // Actually important.
}

I might not ever want to have to care that DoAJigOn with whatever it does to Database is a possibility, because package dancelibrary is externally defined and owned, making my test fragile or overly specified.