verifyInOrder AND verify count

[KyleFin]

Most times when I use verifyInOrder, I'd like to also verify that the specified invocations are the only invocations. As the docs say, [verifyInOrder] only verifies that each call was made in the order given, but not that those were the only calls. That makes sense, but then my next thought (every time) is to also verify the number of calls:

// DOES NOT WORK AS HOPED
verify(() => myMethod(any())).called(2);
verifyInOrder([() => myMethod(1), () => myMethod(5)]);

This doesn't work because (according to verify doc comment) When mocktail verifies a method call, said call is then excluded from further verifications. A single method call cannot be verified from multiple calls to verify, or verifyInOrder. See more details in the FAQ. (as an aside, I no longer found details about this in the FAQ. Or I'm not sure where to look).

I'd love to have a way to verify that a specific set of invocations occurred in the correct order and they were the only invocations.

One way I thought to do this is to duplicate the test with one execution verifying the invocation count and the other verifying invocation order. This seemed to work fine for a blocTest:

blocTest<MyBloc, MyState>(
  'invokes method expected number of times when provoked',
  build: () => MyBloc(),
  act: (bloc) => bloc.add(ProvokedEvent()),
  verify: (_) {
    verify(() => myMethod(any())).called(2);
  },
);

blocTest<MyBloc, MyState>(
  'invokes method in expected order when provoked',
  build: () => MyBloc(),
  act: (bloc) => bloc.add(ProvokedEvent()),
  verify: (_) {
    verifyInOrder([() => myMethod(1), () => myMethod(5)]);
  },
);

It would be ideal to have both verifications in one test execution, but I feel like it seems fine in many situations to execute a lightweight test twice to more accurately verify behavior is as intended.

This approach made me want to share the test naming and code to make the test cases more readable AND ensure if one changes the other changes too.

One way to accomplish this could be something like flutter_test's test variants (this would be trickier than I expected because TestVariant isn't part of dart_test):

class _VerificationVariant {
  const _VerificationVariant(this.description, this.verification);

  final String description;
  final Function verification;
}

final _variants = ValueVariant(const {
  _VerificationVariant(
    'correct number of times',
    () => verify(() => myMethod(any())).called(2),
  ),
  _VerificationVariant(
    'in correct order',
    () => verifyInOrder([() => myMethod(1), () => myMethod(5)]),
  ),
});

blocTest<MyBloc, MyState>(
  'invokes method ${_variants.currentValue.description} when provoked',
  build: () => MyBloc(),
  act: (bloc) => bloc.add(ProvokedEvent()),
  verify: (_) {
    _variants.currentValue.verification();
  },
  variants: _variants
);

// OUTPUT: Test executes twice (once for each _VerificationVariant in _variants.
// invokes method correct number of times when provoked
// invokes method in correct order when provoked

I'm not sure if it might make sense to add a wrapper around Dart test or blocTest that has a variants or make a way you could pass a list to blocTest.verify or something like that. As I'm writing this I'm realizing there are probably tons or edge cases to consider, but it would be nice to write a test once and use multiple verify statements.

[felangel]

Hi @KyleFin 👋 Thanks for opening an issue!

Have you tried using verifyNoMoreInteractions? It sounds like it's exactly what you're looking for. You can use it in conjunction with verifyInOrder to ensure interactions occur in that exact order and that no other interactions occur.

Let me know if that helps 👍

[KyleFin]

Hey @felangel! :)

verifyNoMoreInteractions looks useful, but not quite what I'm looking for. If I'm understanding correctly, verifyNoMoreInteractions can be used to verify there are no invocations AFTER those verified by verifyInOrder. This is an important case, but I'm more concerned about having unexpected invocations between those I'm verifying with verifyInOrder. I may also want to assert there were no calls before those I'm expecting. Or there may be other invocations that are acceptable, but I want to verify that a specific method was invoked in a precise sequence.

Some examples using verifyInOrder AND verify().called() how I wish I could use them:

class MyClass {
  void methodToTest(int i);
  void ignoreMe();
}

class MockMyClass extends Mock implements MyClass {}

// TESTS
// All verifyInOrder and verify calls are identical.
// What changes is which methods are invoked on mockObject during test.
final mockObject = MockMyClass();

// CASE 1 (Should pass)
test('Invocation before that can be ignored', () {
  mockObject
      ..ignoreMe()
      ..methodToTest(1)
      ..methodToTest(2);
  verifyInOrder([() => mockObject.methodToTest(1), () => mockObject.methodToTest(2)]);
  verify(() => mockObject.methodToTest(any()).called(2);
});

// CASE 2 (Should pass)
test('Invocation between that can be ignored', () {
  mockObject
      ..methodToTest(1)
      ..ignoreMe()
      ..methodToTest(2);
  verifyInOrder([() => mockObject.methodToTest(1), () => mockObject.methodToTest(2)]);
  verify(() => mockObject.methodToTest(any()).called(2);
});

// CASE 3 (Should pass. verifyNoMoreInteractions won't work here.)
test('Invocation after that can be ignored', () {
  mockObject
      ..methodToTest(1)
      ..methodToTest(2)
      ..ignoreMe();
  verifyInOrder([() => mockObject.methodToTest(1), () => mockObject.methodToTest(2)]);
  verify(() => mockObject.methodToTest(any()).called(2);
});

// CASE 4 (Should fail)
test('Improper invocation before', () {
  mockObject
      ..methodToTest(0)
      ..methodToTest(1)
      ..methodToTest(2);
  verifyInOrder([() => mockObject.methodToTest(1), () => mockObject.methodToTest(2)]);
  verify(() => mockObject.methodToTest(any()).called(2);
});

// CASE 5 (Should fail)
test('Improper invocation between', () {
  mockObject
      ..methodToTest(1)
      ..methodToTest(0)
      ..methodToTest(2);
  verifyInOrder([() => mockObject.methodToTest(1), () => mockObject.methodToTest(2)]);
  verify(() => mockObject.methodToTest(any()).called(2);
});

// CASE 6 (Should fail)
test('Improper invocation after', () {
  mockObject
      ..methodToTest(1)
      ..methodToTest(2)
      ..methodToTest(0);
  verifyInOrder([() => mockObject.methodToTest(1), () => mockObject.methodToTest(2)]);
  verify(() => mockObject.methodToTest(any()).called(2);
});

[KyleFin]

The first solution I thought of was to just run the test twice (once for order, once for count), but maybe there's a way we could store a count of the invocations before they're verified and then verify the count later?