ThrowTheSwitch / CMock

CMock - Mock/stub generator for C
http://throwtheswitch.org
MIT License
683 stars 274 forks source link

[BUG?] Ignore not working as expected #480

Open parmi93 opened 2 months ago

parmi93 commented 2 months ago
//foo.h
void funA(int a, int b);
void funB(char a);
//bar.c
#include "foo.h"
void my_function()
{
    funA(0, 0);
    funB('x');
    funA(1, 1);
    funA(2, 2);
}
//test_my_function.c
#include "bar.h"
#include "mock_foo.h"

void test_my_function()
{
    funA_Expect(0, 0);
    funB_Expect('x');
    funA_Ignore(); // I expect any further calls to the funA function to be ignored.

    my_function();
}

The mock code auto-generated is:

void funA_CMockIgnore(void)
{
  Mock.funA_IgnoreBool = (char)1;
}

void funA_CMockStopIgnore(void)
{
  Mock.funA_IgnoreBool = (char)0;
}

Test is failing with the following message:

-------------------
FAILED TEST SUMMARY
-------------------
[test_my_function.c]
  Test: test_my_function
  At line (90): "Function funB.  Called earlier than expected."
Letme commented 2 months ago

Ignore, ignores all. So its not what your comment says.

mvandervoord commented 2 months ago

Hi @parmi93

As @Letme explained, the Ignore function doesn't work as you think it does. It's not queueing up the ability to ignore a function call AFTER it was expected, it's replacing the expectation with a "nevermind... you can ignore any calls to this function".

At first that might seem like a silly choice. But consider the opposite direction:

    funA_Ignore(); 
    funA_Expect(0, 0);
    funB_Expect('x');

If we use the same logic as our actual implementation, this is easy... the Ignore directive is replaced with a "actually, I want you to check for funcA to get called" and the Expectation makes sense.

If we accepted that it would be queued, as your original assumption was, then what does this mean? Does it need to ignore a particular number of calls before the Expect (which defeats the purpose of ignore)? Does it only want to check the arguments of the last time funcA gets called? (We'd have no way of knowing it's the last). etc.

So, in attempt to keep the syntax consistent, ignore is turned on/off by overrides. :)

parmi93 commented 2 months ago

Thanks for the clarification, I get what you mean.

I was fooled by a superficial reading of this section of the documentation: https://github.com/ThrowTheSwitch/CMock/blob/9192a950897929d948027421e30a372b1770469b/docs/CMock_Summary.md?plain=1#L165-L176

Specifically:

ignore a particular function for PART OF A TEST but dont want to ignore it later on

This led me to think that funA_Ignore() was also ignoring calls to funA() only for part of the test, but this is not true neither for funA_Ignore() nor for funA_StopIgnore(), both calls to this mock act "globally" for the test, they do not ignore or stop ignoring for part of the test. Is this my misinterpretation of the documentation? Or is there something I'm missing?

mvandervoord commented 2 months ago

Hm. That IS a confusing description.

I guess technically it's true because you can enable and disable the ignore function at different points during a test... but you'd need to actually USE the mocks between those calls.

I'm going to reopen this... we'll either need to improve the docs here or make it work better. :)

parmi93 commented 2 months ago

Perhaps it would be useful to have a mock like funA_StartIgnore() which allows to ignore all calls to funA() from now on.

This way I could change my test to the following:

//test_my_function.c
#include "bar.h"
#include "mock_foo.h"

void test_my_function()
{
    funA_Expect(0, 0);
    funB_Expect('x');
    funA_StartIgnore(); // I expect any further calls to the funA function to be ignored.

    my_function();
}
Letme commented 2 months ago

Just write expects in for loop or something, because how many times do you want that ignore to count until it is stop?

parmi93 commented 2 months ago

Yes, I would like calls to funA() to be ignored until my_function() finishes executing, but starting to ignore calls to funA() only after the call to funB().

Letme commented 2 months ago

But what do you expect to happen in situation:

void test_my_function()
{
    funA_Expect(0, 0);
    funB_Expect('x');
    funA_StartIgnore(); // I expect any further calls to the funA function to be ignored.
    funA_StopIgnore(); // How many calls in between you wanted ignored?
    funA_Expect(1, 1);
}

This is just start of complexity you hit, if you allow ignore after some time. What about when you add complexity of IgnoreAndReturn?

parmi93 commented 2 months ago

But what do you expect to happen in situation:

void test_my_function()
{
    funA_Expect(0, 0);
    funB_Expect('x');
    funA_StartIgnore(); // I expect any further calls to the funA function to be ignored.
    funA_StopIgnore(); // How many calls in between you wanted ignored?
    funA_Expect(1, 1);
}

This is just start of complexity you hit, if you allow ignore after some time. What about when you add complexity of IgnoreAndReturn?

In this situation I would expect that no calls to funA() will be ignored, because funA_StopIgnore() was called immediately after funA_StartIgnore()

edit

Regarding IgnoreAndReturn, we could have a mock like funA_StartIgnoreAndReturn(), which similarly to funA_StartIgnore() ignores all calls to funA() from that moment on and always returning the same value.

Letme commented 2 months ago

Ok, what then do you expect from

void test_my_function()
{
    funA_Expect(0, 0);
    funB_Expect('x');
    funA_StartIgnore(); // I expect any further calls to the funA function to be ignored.
    funB_Expect('x');
    funA_StopIgnore(); // How many calls in between you wanted ignored?
    funA_Expect(1, 1);
}

Maybe you do not realize, but this is not run at the time of the call, but just expects are placed on the stack and then hit as you call your test function, so timing wise both examples should be the same.

parmi93 commented 2 months ago

ohh, I get what you mean, actually what I proposed makes no sense at all! lol

maybe the following solution might work: