Successful verification results are not reported when using Catch

[fgsalomon]

When using Fakeit along Catch Verify successes are not reported.

#define CATCH_CONFIG_MAIN
#include "catch.hpp"
#include "fakeit.hpp"

using namespace fakeit;

struct SomeInterface {
    virtual void foo(int) = 0;
    virtual int bar(int,int) = 0;
    virtual ~SomeInterface() = 0;
};

TEST_CASE("FakeItTest"){

    Mock<SomeInterface> mock;  
    SomeInterface & i = mock.get();

    When(Method(mock,foo)).AlwaysReturn(); 
    When(Method(mock,bar)).AlwaysReturn(0); 

    SECTION("Section"){
        CHECK(i.bar(1,1) == 0);
        Verify(Method(mock,bar)).AtLeast(1);
    }
}

If I execute this test only 1 assertion is reported:

===============================================================================
All tests passed (1 assertion in 1 test case)

But if I change the verification to "AtLeast(2)" 2 assertions are reported:

Verification error
  Expected pattern: mock.bar( Any arguments )
  Expected matches: at least 2
  Actual matches  : 1
  Actual sequence : total of 1 actual invocations:
    mock.bar(1, 1)

===============================================================================
test cases: 1 | 1 failed
assertions: 2 | 1 passed | 1 failed

Catch version: 1.9.6 FakeIt: Generated: 2017-05-07 09:27:02.651812

[fgsalomon]

I've found a "workaround" using CHECK macro from Catch.

CHECK(Verify(Method(mock,bar)).AtLeast(1) ==  true);

I wanted to use:

CHECK(Verify(Method(mock,bar)).AtLeast(1));

but this doesn't compile. The message error is:

/home/fgsalomon/CatchTest/catch.hpp: In instantiation of ‘void Catch::ExpressionLhs<T>::endExpression() [with T = const fakeit::SequenceVerificationProgress::Terminator&]’:
/home/fgsalomon/CatchTest/main.cpp:27:8:   required from here
/home/fgsalomon/CatchTest/catch.hpp:1858:26: error: passing ‘const fakeit::SequenceVerificationProgress::Terminator’ as ‘this’ argument discards qualifiers [-fpermissive]
         m_truthy = m_lhs ? true : false;
                          ^
In file included from /home/fgarcia/Proyectos/CatchTest/main.cpp:3:0:
/home/fgsalomon/CatchTest/fakeit.hpp:8840:13: note:   in call to ‘fakeit::SequenceVerificationProgress::Terminator::operator bool()’
             operator bool() {
             ^

The only issue now is that when the assertion fails it doesn't report the same information. Before:

/home/fgsalomon/CatchTest/main.cpp:27: FAILED:
explicitly with message:
  /home/fgsalomon/CatchTest/main.cpp:27: Verification error
  Expected pattern: mock.bar( Any arguments )
  Expected matches: at least 2
  Actual matches  : 1
  Actual sequence : total of 1 actual invocations:
    mock.bar(1, 1)

And now:

/home/fgsalomon/CatchTest/main.cpp:27: FAILED:
  CHECK( true == Verify( mock.template stub<5>(&std::remove_reference<decltype(mock.get())>::type::bar).setMethodDetails("mock","bar") ).setFileInfo("/home/fgarcia/Proyectos/CatchTest/main.cpp", 27, __func__).AtLeast(2) )
with expansion:
  true == {?}

Any tips?

[eranpeer]

The last commit from Nov-5-2017, supports Catch 2.0.1. It also contains a fix to this issue. Please reopen this issue if you encounter any problem.

[fgsalomon]

I've just tested it using Catch 2.0.1 (Generated: 2017-11-03 11:53:39.642003) and FakeIt's last commit (Generated: 2017-11-05 20:30:40.182814). The issue is still present. Furthermore, the "workaround" using CHECK no longer works.

[eranpeer]

Thanks, I'll try to fix it tomorrow.

On Nov 11, 2017 05:09, "Fran García Salomón" notifications@github.com wrote:

I've just tested it using Catch 2.0.1 (Generated: 2017-11-03 11:53:39.642003) and FakeIt's last commit (Generated: 2017-11-05 20:30:40.182814). The issue is still present. Furthermore, the "workaround" using CHECK no longer works.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/eranpeer/FakeIt/issues/103#issuecomment-343663636, or mute the thread https://github.com/notifications/unsubscribe-auth/ACc8grFVv2KryFMEwBeZ4iE7IKbYsE0jks5s1ZxwgaJpZM4OhJO7 .

[eranpeer]

Ok, I fixed the workaround, It will now work. You can now use CHECK(mock.verify(.....)). Please download the latest build.

As for the issue itself. Fakeit currently only reports verification failures to the Unit Testing framework, as a result the success cases are not listed on the Catch test report. I will issue a fix in the near future, meantime please use the workaround.

On Sat, Nov 11, 2017 at 10:14 AM, Eran Pe'er eranpe@gmail.com wrote:

Thanks, I'll try to fix it tomorrow.

On Nov 11, 2017 05:09, "Fran García Salomón" notifications@github.com wrote:

I've just tested it using Catch 2.0.1 (Generated: 2017-11-03 11:53:39.642003) and FakeIt's last commit (Generated: 2017-11-05 20:30:40.182814). The issue is still present. Furthermore, the "workaround" using CHECK no longer works.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/eranpeer/FakeIt/issues/103#issuecomment-343663636, or mute the thread https://github.com/notifications/unsubscribe-auth/ACc8grFVv2KryFMEwBeZ4iE7IKbYsE0jks5s1ZxwgaJpZM4OhJO7 .

-- Eran 1-424-2504000