search

eranpeer / FakeIt

C++ mocking made easy. A simple yet very expressive, headers only library for c++ mocking.
MIT License
1.22k stars 170 forks source link

Comparing invocation history of two different mocks #331

Open freemin7 opened 2 months ago

freemin7 commented 2 months ago

Motivation

Imagine a situation where you are refactoring code which are supposed to calls mocked methods in some "similar" order with similar parameters.

I checked the library and this equality testing of invocation history is not documented. I imagine something like this pseudo code:

int acceptance_test(){
  Mock<SomeInterface> mock1;
  Mock<SomeInterface> mock2;

  OldAndCrusted reference;
  NewAndUntrusted contestant;

  SomeInterface &reference_mock = mock1.get();
  SomeInterface &contestant_mock = mock2.get();

  // Production code exhaustively testing things
  reference.functionality(reference_mock); 
  contestant.functionality(contestant_mock); 

  // creating history objects
  std::unordered_set<Invocation *> reference_history;
  std::unordered_set<Invocation *> contestant_history;

  // fill history objects
  mock1.getActualInvocations( reference_history );
  mock2.getActualInvocations( contestant_history );

  // Compare histories
  return compareHistoriesDifferenceExplaination(std::cout, reference_history, contestant_history);
}

I am still unfamiliar with FakeIt and recently started contributing to a code base which uses it.

Questions

In my case "similar" means identical. But i could see an extension of this to cover mocked calls having equality relations, properties like idem-potency and information whether certain calls commute given certain arguments being interesting to pursue further given an use case.

FranckRJ commented 2 months ago

Are there philosophical reasons why this use of FakeIt would be discouraged/"bad"?

I'm not sure that letting the user have access to the invocation history is a good idea. I know they can already do it because the functions you mentioned are part of the public API so they're accessible, but as far as I know it's not documented anywhere so it's not really part of the "official" API.

My approach would be to give the user the minimal API surface needed to do what they want to do, so it would rather look like this: mock1.verifyHasSameInvocationHistoryHas(mock2), or fakeit::VerifyHasSameInvocationHistory(mock1, mock2).

But maybe I'm wrong and we can safely make getActualInvocations part of the "official" API (by documenting it) and provide a bunch of methods to manipulate and compare invocations so the user can do what they want, I haven't looked too much into it so I have no idea how easy it is to make it safe.

Does FakeIt or a different library extending it already provide convenience functions which could help implement compareHistoriesDifferenceExplaination?

Not that I'm aware of.

Are there any obvious problems with this approach which will complicate things, such as edge cases around comparing invocations of different Mocks of mocking the same class called by different classes?

I haven't really looked at how invocation data are stored so I don't know. One issue that you may encounter is the one of dangling references (https://github.com/eranpeer/FakeIt/issues/274). With the standard Verify you can work around this (as explained in the issue) but because VerifyHasSameInvocationHistory / compareHistoriesDifferenceExplaination is a single non-customizable function, you won't be able to change how a specific invocation is compared with another, so the second workaround won't be applicable (and the first one is situational, if your function should accept different arguments you can really use it).

Maybe you can ignore that issue. Maybe you can try a different approach by comparing the identity of referenced objects (that they have the same address) instead of their equality, but that would differ from how Verify works so maybe it's not a good idea, I don't know (and it doesn't guarantee that it was the same object anyway).

Is there any functionality or internal methods i should be aware of when implementing compareHistoriesDifferenceExplaination?

Not that I'm aware of.

Would compareHistoriesDifferenceExplaination under a less unwieldy name be an contribution the community appreciates?

If it's not too complicated to implement I think it would be a good idea. By "too complicated" I'm mainly talking about compilation time, so if it requires dozen of templated templates of templates I'll be more cautious about merging that feature (I'll wait to see if other people want it merged), but I don't think it will be to complicated so it shouldn't be an issue.

If so how should this contribution be distributed? Does it have a place in the core FakeIt library or should be in a separate library?

You can submit a PR, the worst case scenario is that you get some feedback on your implementation, and then you'll always be able to create an "add-on" library anyway.