search

eranpeer / FakeIt

C++ mocking made easy. A simple yet very expressive, headers only library for c++ mocking.
MIT License
1.24k stars 170 forks source link

Default all methods to "Fake it" instead of throw #82

Open helmesjo opened 7 years ago

helmesjo commented 7 years ago

Hi!

Basically, as I understand it, when creating a mock I have to explicitly do Fake(Method(,)) for each method that might get called during testing. Currently I'm working on some legacy code that does alot in alot of places, basically forcing me to Fake each method explicitly to even get past the method being tested.

Could it be possible to, by default by choice, make all invocations be faked/ignored/return defaults ?

This is how most other mocking-frameworks behave coming from .Net, (even Isolator++ behave like this to my knowledge).

Would remove alot of boilerplate code!

eranpeer commented 7 years ago

Hi, Unfortunately, this is a limitation of the language. In order to properly Fake a method Fakeit must know it's prototype. Currently C++ does not support "reflection" and there's no way I can list all the methods of a class in order to fake them. When you Fake a method you actually tell Fakeit about the exact prototype and only then Fakeit can inject the appropriate default behavior. Maybe a future release of c++ will support reflection. If so, I will add this important feature. Thanks,

On Mar 9, 2017 6:38 PM, "Fred Helmesjö" notifications@github.com wrote:

Hi!

Basically, as I understand it, when creating a mock I have to explicitly do Fake(Method(,)) for each method that might get called during testing. Currently I'm working on some legacy code that does alot in alot of places, basically forcing me to Fake each method explicitly to even get past the method being tested.

Could it be possible to, by default by choice, make all invocations be faked/ignored/return defaults ?

This is how most other mocking-frameworks behave coming from .Net, (even Isolator++ behave like this to my knowledge).

Would remove alot of boilerplate code!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/eranpeer/FakeIt/issues/82, or mute the thread https://github.com/notifications/unsubscribe-auth/ACc8gtB3yUrW78HBtKSN_jPhSwBA-jlKks5rkCr2gaJpZM4MYU65 .

helmesjo commented 7 years ago

Yeah, the absence of reflection is something one notices quickly coming from a .Net-background! :) At the same time one keeps getting amazed by the "magic" possible with Macro-nesting though!

Reading through Isolator++ docs, however, I noticed them having this feature (returning default values). How they've implemented their mocks, and what they really do in the background to allow this I don't know (haven't even tried their free trial), but it works.

See here:

class MyPureClass
{
public:
   virtual int GetResult() = 0;
}
// ...
MyPureClass* fakeMyClass = FAKE<MyPureClass>();

[..] This fakes all the methods on the MyClass class. Primitives return 0, and methods returning pointers, return pointers to fake objects.

  This is pretty much similar to how mocking frameworks in other languages work.

helmesjo commented 7 years ago

Look also here at "Recursive Fakes" (just like in say NSubstitute).

eranpeer commented 7 years ago

Yea, I'm aware there are solutions in java & c# (and also in Isolator++), I just can't find a way to implement this one. I'll be happy if someone will submit a Pull Request on this issue.

mfontanini commented 6 years ago

Is the default behavior to throw now? It seems to crash for me. e.g. this code:

#define CATCH_CONFIG_MAIN
#include <catch2/catch.hpp>
#include <catch/fakeit.hpp>
#include <string>

class Foo {
public:
    virtual ~Foo() = default;
    virtual std::string bar() = 0;
};

TEST_CASE("test") {
    fakeit::Mock<Foo> mocked;
    //fakeit::Fake(Method(mocked, bar)); < this fixes it
    mocked.get().bar();
}

Blows up. Valgrind output:

==15726==    at 0x44B4F8: fakeit::VirtualTableBase::getCookie(int) (fakeit.hpp:5678)
==15726==    by 0x48215D: fakeit::MockImpl<Foo<> >::getMockImpl(void*) (fakeit.hpp:8158)
==15726==    by 0x47B128: fakeit::MockImpl<Foo<> >::unmocked() (fakeit.hpp:8169)
==15726==    by 0x42BF9B: ____C_A_T_C_H____T_E_S_T____0() (test.cpp:15)
==15726==    by 0x41B157: Catch::TestInvokerAsFunction::invoke() const (catch.hpp:10331)
==15726==    by 0x41A7FC: Catch::TestCase::invoke() const (catch.hpp:10232)
==15726==    by 0x4151ED: Catch::RunContext::invokeActiveTestCase() (catch.hpp:9104)
==15726==    by 0x414F01: Catch::RunContext::runCurrentTest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (catch.hpp:9078)
==15726==    by 0x413ABF: Catch::RunContext::runTest(Catch::TestCase const&) (catch.hpp:8861)
==15726==    by 0x416710: Catch::(anonymous namespace)::runTests(std::shared_ptr<Catch::Config> const&) (catch.hpp:9394)
==15726==    by 0x417A4C: Catch::Session::runInternal() (catch.hpp:9592)
==15726==    by 0x4177DA: Catch::Session::run() (catch.hpp:9549)
==15726== 
==15726== Invalid read of size 8
==15726==    at 0x44B4F8: fakeit::VirtualTableBase::getCookie(int) (fakeit.hpp:5678)
==15726==    by 0x48215D: fakeit::MockImpl<Foo<> >::getMockImpl(void*) (fakeit.hpp:8158)
==15726==    by 0x47B128: fakeit::MockImpl<Foo<> >::unmocked() (fakeit.hpp:8169)
==15726==    by 0x42BF9B: ____C_A_T_C_H____T_E_S_T____0() (test.cpp:15)
==15726==    by 0x41B157: Catch::TestInvokerAsFunction::invoke() const (catch.hpp:10331)
==15726==    by 0x41A7FC: Catch::TestCase::invoke() const (catch.hpp:10232)
==15726==    by 0x4151ED: Catch::RunContext::invokeActiveTestCase() (catch.hpp:9104)
==15726==    by 0x414F01: Catch::RunContext::runCurrentTest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (catch.hpp:9078)
==15726==    by 0x413ABF: Catch::RunContext::runTest(Catch::TestCase const&) (catch.hpp:8861)
==15726==    by 0x416710: Catch::(anonymous namespace)::runTests(std::shared_ptr<Catch::Config> const&) (catch.hpp:9394)
==15726==    by 0x417A4C: Catch::Session::runInternal() (catch.hpp:9592)
==15726==    by 0x4177DA: Catch::Session::run() (catch.hpp:9549)
==15726==  Address 0xffffffffffffffe0 is not stack'd, malloc'd or (recently) free'd
eranpeer commented 6 years ago

The default behavior was always to throw an “unmocked exception”. You must stub the methods either by Fake(Method(mocked, bar)) or by When(Method(mocked, bar)). Though it looks like that instead of getting the expected exception you got a crash. I will look into it. Thanks,

Sent from Mail for Windows 10

From: Matias Fontanini Sent: Friday, August 24, 2018 10:42 AM To: eranpeer/FakeIt Cc: eranpeer; Comment Subject: Re: [eranpeer/FakeIt] Default all methods to "Fake it" instead ofthrow (#82)

Is the default behavior to throw now? It seems to crash for me. e.g. this code:

define CATCH_CONFIG_MAIN

include <catch2/catch.hpp>

include <catch/fakeit.hpp>

include

class Foo { public: virtual ~Foo() = default; virtual std::string bar() = 0; };

TEST_CASE("test") { fakeit::Mock mocked; //fakeit::Fake(Method(mocked, bar)); < this fixes it mocked.get().bar(); } Blows up. Valgrind output: ==15726== at 0x44B4F8: fakeit::VirtualTableBase::getCookie(int) (fakeit.hpp:5678) ==15726== by 0x48215D: fakeit::MockImpl<Foo<> >::getMockImpl(void) (fakeit.hpp:8158) ==15726== by 0x47B128: fakeit::MockImpl<Foo<> >::unmocked() (fakeit.hpp:8169) ==15726== by 0x42BF9B: C_A_T_C_HT_E_S_T__0() (test.cpp:15) ==15726== by 0x41B157: Catch::TestInvokerAsFunction::invoke() const (catch.hpp:10331) ==15726== by 0x41A7FC: Catch::TestCase::invoke() const (catch.hpp:10232) ==15726== by 0x4151ED: Catch::RunContext::invokeActiveTestCase() (catch.hpp:9104) ==15726== by 0x414F01: Catch::RunContext::runCurrentTest(std::cxx11::basic_string<char, std::char_traits, std::allocator >&, std::__cxx11::basic_string<char, std::char_traits, std::allocator >&) (catch.hpp:9078) ==15726== by 0x413ABF: Catch::RunContext::runTest(Catch::TestCase const&) (catch.hpp:8861) ==15726== by 0x416710: Catch::(anonymous namespace)::runTests(std::shared_ptr const&) (catch.hpp:9394) ==15726== by 0x417A4C: Catch::Session::runInternal() (catch.hpp:9592) ==15726== by 0x4177DA: Catch::Session::run() (catch.hpp:9549) ==15726== ==15726== Invalid read of size 8 ==15726== at 0x44B4F8: fakeit::VirtualTableBase::getCookie(int) (fakeit.hpp:5678) ==15726== by 0x48215D: fakeit::MockImpl<Foo<> >::getMockImpl(void) (fakeit.hpp:8158) ==15726== by 0x47B128: fakeit::MockImpl<Foo<> >::unmocked() (fakeit.hpp:8169) ==15726== by 0x42BF9B: C_A_T_C_HT_E_S_T__0() (test.cpp:15) ==15726== by 0x41B157: Catch::TestInvokerAsFunction::invoke() const (catch.hpp:10331) ==15726== by 0x41A7FC: Catch::TestCase::invoke() const (catch.hpp:10232) ==15726== by 0x4151ED: Catch::RunContext::invokeActiveTestCase() (catch.hpp:9104) ==15726== by 0x414F01: Catch::RunContext::runCurrentTest(std::cxx11::basic_string<char, std::char_traits, std::allocator >&, std::__cxx11::basic_string<char, std::char_traits, std::allocator >&) (catch.hpp:9078) ==15726== by 0x413ABF: Catch::RunContext::runTest(Catch::TestCase const&) (catch.hpp:8861) ==15726== by 0x416710: Catch::(anonymous namespace)::runTests(std::shared_ptr const&) (catch.hpp:9394) ==15726== by 0x417A4C: Catch::Session::runInternal() (catch.hpp:9592) ==15726== by 0x4177DA: Catch::Session::run() (catch.hpp:9549) ==15726== Address 0xffffffffffffffe0 is not stack'd, malloc'd or (recently) free'd — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

eranpeer commented 6 years ago

I can't reproduce that. Can you please share compiler version + all compilation flags. Thanks,

On Fri, Aug 24, 2018 at 12:17 PM Eran Pe'er eranpe@gmail.com wrote:

The default behavior was always to throw an “unmocked exception”.

You must stub the methods either by Fake(Method(mocked, bar)) or by When(Method(mocked, bar)).

Though it looks like that instead of getting the expected exception you got a crash.

I will look into it.

Thanks,

Sent from Mail https://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10

From: Matias Fontanini notifications@github.com Sent: Friday, August 24, 2018 10:42 AM To: eranpeer/FakeIt FakeIt@noreply.github.com Cc: eranpeer eranpe@gmail.com; Comment comment@noreply.github.com Subject: Re: [eranpeer/FakeIt] Default all methods to "Fake it" instead ofthrow (#82)

Is the default behavior to throw now? It seems to crash for me. e.g. this code:

define CATCH_CONFIG_MAIN

include <catch2/catch.hpp>

include <catch/fakeit.hpp>

include

class Foo {

public:

virtual ~Foo() = default;

virtual std::string bar() = 0;

};

TEST_CASE("test") {

fakeit::Mock<Foo> mocked;

//fakeit::Fake(Method(mocked, bar)); < this fixes it

mocked.get().bar();

}

Blows up. Valgrind output:

==15726== at 0x44B4F8: fakeit::VirtualTableBase::getCookie(int) (fakeit.hpp:5678)

==15726== by 0x48215D: fakeit::MockImpl<Foo<> >::getMockImpl(void*) (fakeit.hpp:8158)

==15726== by 0x47B128: fakeit::MockImpl<Foo<> >::unmocked() (fakeit.hpp:8169)

==15726== by 0x42BF9B: C_A_T_C_HT_E_S_T____0() (test.cpp:15)

==15726== by 0x41B157: Catch::TestInvokerAsFunction::invoke() const (catch.hpp:10331)

==15726== by 0x41A7FC: Catch::TestCase::invoke() const (catch.hpp:10232)

==15726== by 0x4151ED: Catch::RunContext::invokeActiveTestCase() (catch.hpp:9104)

==15726== by 0x414F01: Catch::RunContext::runCurrentTest(std::cxx11::basic_string<char, std::char_traits, std::allocator >&, std::cxx11::basic_string<char, std::char_traits, std::allocator >&) (catch.hpp:9078)

==15726== by 0x413ABF: Catch::RunContext::runTest(Catch::TestCase const&) (catch.hpp:8861)

==15726== by 0x416710: Catch::(anonymous namespace)::runTests(std::shared_ptr const&) (catch.hpp:9394)

==15726== by 0x417A4C: Catch::Session::runInternal() (catch.hpp:9592)

==15726== by 0x4177DA: Catch::Session::run() (catch.hpp:9549)

==15726==

==15726== Invalid read of size 8

==15726== at 0x44B4F8: fakeit::VirtualTableBase::getCookie(int) (fakeit.hpp:5678)

==15726== by 0x48215D: fakeit::MockImpl<Foo<> >::getMockImpl(void*) (fakeit.hpp:8158)

==15726== by 0x47B128: fakeit::MockImpl<Foo<> >::unmocked() (fakeit.hpp:8169)

==15726== by 0x42BF9B: C_A_T_C_HT_E_S_T____0() (test.cpp:15)

==15726== by 0x41B157: Catch::TestInvokerAsFunction::invoke() const (catch.hpp:10331)

==15726== by 0x41A7FC: Catch::TestCase::invoke() const (catch.hpp:10232)

==15726== by 0x4151ED: Catch::RunContext::invokeActiveTestCase() (catch.hpp:9104)

==15726== by 0x414F01: Catch::RunContext::runCurrentTest(std::cxx11::basic_string<char, std::char_traits, std::allocator >&, std::cxx11::basic_string<char, std::char_traits, std::allocator >&) (catch.hpp:9078)

==15726== by 0x413ABF: Catch::RunContext::runTest(Catch::TestCase const&) (catch.hpp:8861)

==15726== by 0x416710: Catch::(anonymous namespace)::runTests(std::shared_ptr const&) (catch.hpp:9394)

==15726== by 0x417A4C: Catch::Session::runInternal() (catch.hpp:9592)

==15726== by 0x4177DA: Catch::Session::run() (catch.hpp:9549)

==15726== Address 0xffffffffffffffe0 is not stack'd, malloc'd or (recently) free'd

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/eranpeer/FakeIt/issues/82#issuecomment-415831049, or mute the thread https://github.com/notifications/unsubscribe-auth/ACc8gmIiTV25KqwxDcb4FTqEgfO1dajKks5uUDsTgaJpZM4MYU65 .[image: https://github.com/notifications/beacon/ACc8ghaTJAsAvmDDRbmBdYDWMANE48Mpks5uUDsTgaJpZM4MYU65.gif]

-- Eran 1-424-2504000

eranpeer commented 6 years ago

Found it, I was compiling with -O0. When I changed to -O1 it reproduced the crash. For now you can use -O0, it should work.

On Sun, Aug 26, 2018 at 3:27 PM Eran Pe'er eranpe@gmail.com wrote:

I can't reproduce that. Can you please share compiler version + all compilation flags. Thanks,

On Fri, Aug 24, 2018 at 12:17 PM Eran Pe'er eranpe@gmail.com wrote:

The default behavior was always to throw an “unmocked exception”.

You must stub the methods either by Fake(Method(mocked, bar)) or by When(Method(mocked, bar)).

Though it looks like that instead of getting the expected exception you got a crash.

I will look into it.

Thanks,

Sent from Mail https://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10

From: Matias Fontanini notifications@github.com Sent: Friday, August 24, 2018 10:42 AM To: eranpeer/FakeIt FakeIt@noreply.github.com Cc: eranpeer eranpe@gmail.com; Comment comment@noreply.github.com Subject: Re: [eranpeer/FakeIt] Default all methods to "Fake it" instead ofthrow (#82)

Is the default behavior to throw now? It seems to crash for me. e.g. this code:

define CATCH_CONFIG_MAIN

include <catch2/catch.hpp>

include <catch/fakeit.hpp>

include

class Foo {

public:

virtual ~Foo() = default;

virtual std::string bar() = 0;

};

TEST_CASE("test") {

fakeit::Mock<Foo> mocked;

//fakeit::Fake(Method(mocked, bar)); < this fixes it

mocked.get().bar();

}

Blows up. Valgrind output:

==15726== at 0x44B4F8: fakeit::VirtualTableBase::getCookie(int) (fakeit.hpp:5678)

==15726== by 0x48215D: fakeit::MockImpl<Foo<> >::getMockImpl(void*) (fakeit.hpp:8158)

==15726== by 0x47B128: fakeit::MockImpl<Foo<> >::unmocked() (fakeit.hpp:8169)

==15726== by 0x42BF9B: C_A_T_C_HT_E_S_T____0() (test.cpp:15)

==15726== by 0x41B157: Catch::TestInvokerAsFunction::invoke() const (catch.hpp:10331)

==15726== by 0x41A7FC: Catch::TestCase::invoke() const (catch.hpp:10232)

==15726== by 0x4151ED: Catch::RunContext::invokeActiveTestCase() (catch.hpp:9104)

==15726== by 0x414F01: Catch::RunContext::runCurrentTest(std::cxx11::basic_string<char, std::char_traits, std::allocator >&, std::cxx11::basic_string<char, std::char_traits, std::allocator >&) (catch.hpp:9078)

==15726== by 0x413ABF: Catch::RunContext::runTest(Catch::TestCase const&) (catch.hpp:8861)

==15726== by 0x416710: Catch::(anonymous namespace)::runTests(std::shared_ptr const&) (catch.hpp:9394)

==15726== by 0x417A4C: Catch::Session::runInternal() (catch.hpp:9592)

==15726== by 0x4177DA: Catch::Session::run() (catch.hpp:9549)

==15726==

==15726== Invalid read of size 8

==15726== at 0x44B4F8: fakeit::VirtualTableBase::getCookie(int) (fakeit.hpp:5678)

==15726== by 0x48215D: fakeit::MockImpl<Foo<> >::getMockImpl(void*) (fakeit.hpp:8158)

==15726== by 0x47B128: fakeit::MockImpl<Foo<> >::unmocked() (fakeit.hpp:8169)

==15726== by 0x42BF9B: C_A_T_C_HT_E_S_T____0() (test.cpp:15)

==15726== by 0x41B157: Catch::TestInvokerAsFunction::invoke() const (catch.hpp:10331)

==15726== by 0x41A7FC: Catch::TestCase::invoke() const (catch.hpp:10232)

==15726== by 0x4151ED: Catch::RunContext::invokeActiveTestCase() (catch.hpp:9104)

==15726== by 0x414F01: Catch::RunContext::runCurrentTest(std::cxx11::basic_string<char, std::char_traits, std::allocator >&, std::cxx11::basic_string<char, std::char_traits, std::allocator >&) (catch.hpp:9078)

==15726== by 0x413ABF: Catch::RunContext::runTest(Catch::TestCase const&) (catch.hpp:8861)

==15726== by 0x416710: Catch::(anonymous namespace)::runTests(std::shared_ptr const&) (catch.hpp:9394)

==15726== by 0x417A4C: Catch::Session::runInternal() (catch.hpp:9592)

==15726== by 0x4177DA: Catch::Session::run() (catch.hpp:9549)

==15726== Address 0xffffffffffffffe0 is not stack'd, malloc'd or (recently) free'd

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/eranpeer/FakeIt/issues/82#issuecomment-415831049, or mute the thread https://github.com/notifications/unsubscribe-auth/ACc8gmIiTV25KqwxDcb4FTqEgfO1dajKks5uUDsTgaJpZM4MYU65 .[image: https://github.com/notifications/beacon/ACc8ghaTJAsAvmDDRbmBdYDWMANE48Mpks5uUDsTgaJpZM4MYU65.gif]

-- Eran 1-424-2504000

-- Eran 1-424-2504000

mfontanini commented 6 years ago

FakeIt has some nice syntax but it seems to be extremely unstable. I understand that's because of how it works under the hood but I think I'll stick to some other mocking library where crashing is not common behavior under any compiler flags.

ericlemes commented 6 years ago

@mfontanini, let me know if you find anything that suits your needs. I tried a lot of different. The most stable seems to be GMock but also has it's drawbacks.

I think with some small improvements in Fake It (I was thinking if would be possible to create some sort of self validation to point out unsupported compiler flags, via static assert or even self testing) to simplify these issues. That's why I've been going down this route of trying to improve Fake It instead of moving away. It is really hard to find a good, stable and supported mocking framework for C++.

For me after one fix, it is definitely working very well.

mfontanini commented 6 years ago

@ericlemes I've been using trompeloeil which has been working great so far, although I have just started using it so it may be too soon to say. I really like how FakeIt looks like but the fact that it just blows up when anything goes wrong is really worrying. How do you know the compiler won't start doing something that breaks FakeIt even when optimization is disabled in the future? In fact:

@eranpeer, building with -O0 doesn't fix this. I'm using g++ 5.4 (default on Ubuntu Xenial) and it still blows up. I'm only passing -O0 -std=c++14 as command line arguments when building that code.