CMock out of memory with ExpectAndReturn

[ngsann]

I am testing some function over a large range of values by using a loop and the test fails with "CMock has run out of memory. Please allocate more." Below is the portion of the code that fails the test (Some part like the assert has been removed for this post. )

for (uint32_t i = 0; i < 0x00FFFF; ++i)
{
      myFunc_ExpectAndReturn(i);
}

Currently, I work around this with StubWithCallback but I was wondering what should be the correct way to test multiple ExpectAndReturn call without chaining the ExpectAndReturn or how do I free the memory ?

[mvandervoord]

The simplest way to test is through an iterator, much like you are doing. The only thing you will probably want to add is a note to CMock to let it know that you want it to verify that all the expectations have been met and that you are starting over with the next iteration. That goes like this:

for (uint32_t i = 0; i < 0x00FFFF; ++i)
{
      myFunc_ExpectAndReturn(i);
      funcBeingTested();
      TEST_ASSERT_MESSAGE("Test whatever else you want");

      resetTest(); // <---- This is generated by the test runner generator script. 
}

[farhangj]

Hi there, Thanks for providing this solution.

I am interested to know why CMOCK can't automatically release memory after the expectations are met in each iteration of the loop.

On a separate note adding resetTest() causes a compiler warning, what do I need to include in my test to get rid of this?

warning: implicit declaration of function 'resetTest' [-Wimplicit-function-declaration] resetTest(); // <---- This is generated by the test runner generator script.

Thanks,

[mvandervoord]

After your #includes in your test module, you can just add a function prototype like the following:

void resetTest(void);

Then it won't complain about that anymore.

CMock COULD release memory during each mock call, but it would slow things down a lot. At the end of each mock call it would have to check to see if there is anything else pending for any mock function. If the answer is no, then it could clear the memory. This would have a large impact on overall performance.

The second issue is validation. The resetTest doesn't just clear memory, it verifies that ALL of the conditions stated so far have been met. If they haven't it fails. This isn't something that can be implied in a test. There is no way for CMock to know when you believe you are done with your test.

[sandeepzac]

Hi Mark,

I have been working with Ceedling for the past few months.

I have a situation where the for loop comes in between a large function under test.

So I have few Expect calls before and after the for loop, and the loop throws the memory error due to 650+ iterations. How do you think I should be approaching the mocks / unit tests in this case?

void test_function(void) {

Expect_calls();

for loop { Expect_calls(); resetTest(); --> doing this does not seem to work. }

Expect_calls(); TEST_ASSERT();

}

Please let me know your thoughts.

Thanks, Sandeep.

[mvandervoord]

Sandeep,

Where in this scenario is your function being tested get called? Does it get called for every loop also, or does the function being tested get called only once after all the expect calls?

If it's being called once per loop, then having resetTest in the loop is the right way to go. We'll want to figure out why it's not working for you, but it's the correct strategy.

If the function only gets called once after the loop, then we don't want resetTest in the loop. This is because resetTest is going to verify that all expectations have been met by the time it is called... since the actual function-under-test has NOT been called yet, it will fail.

So the question becomes "how do we keep it from queueing up 650+ expected function calls?" Myself, I would use _StubWithCallback instead of an _Expect. You can count the number of iterations and use TEST_ASSERT_EQUAL(650, num_calls) at the end of the test. The stub can also perform whatever argument checking you may need. Best of all, it'll avoid queueing up all that data.

I hope this helps!

[sandeepzac]

Hello Mark,

Thanks for your suggestions and inputs.

My function under test gets called only once after all expect calls. So this then rules out resetTest() being used inside the loop. Thanks.

However the actual function under test has a static variable that gets incremented every time this loop is executed and there is a condition within the loop which checks if the static variable is greater than 650.

I am trying to hit this condition for a 100% coverage as well. There are other if conditions inside the loop which forces me to use Expect_ReturnThruPtr - to set certain variables so that these conditions are hit and the incrementation happens as well.

I know this might not be the best designed code, but at this point ill have to work without any changes to the function under test.

You think there is a way to avoid queueing 650+ iterations to achieve this?

I hope my explanation is clear.

Thanks, Sandeep.

[mvandervoord]

Yes, I agree. Don't change the actual source code. Just change the test.

The option I listed above is one way to do this. It still uses CMock to test. But instead of testing it using the _Expect and _ReturnThruPtr calls over and over, you would have it use _StubWithCallback. This would allow you to pass your own verification Stub function to CMock so that it can be used to count the iterations.

For example, let's say the function you are testing looked something like this:

void FunctionUnderTest(void)
{
    int i, all_done;

    OpeningStuff();
    for (i = 0; i < 650; i++) {
        DoSomeStuff( &all_done );
        if (all_done)
            return;
    }

    HandleError();
}

It's quite contrived, but I think it has the components you discussed. There is a function DoSomeStuff that you would have a series of expectations and _ReturnThruPtr calls. Most tests would eventually set all_done to true, allowing the test to end early. You now want to write a test that lets the loop run 650 times so that we can verify that HandleError gets called after.

Because we don't have enough memory to handle those 650 calls in our queue, we choose to use a stub instead:

void Fake_DoSomeStuff(int* all_done, int times_called)
{
  TEST_ASSERT_TRUE_MESSAGE( (times_called <= 650), "We Should Never Reach This Point!");
  *all_done = 0;
}

void test_FunctionUnderTest_should_AllowUsToRun650TimersBeforeGivingUp(void)
{
  OpeningStuff_Expect();
  DoSomeStuff_StubWithCallback( Fake_DoSomeStuff );
  HandleError_Expect();

  FunctionUnderTest();
}

The Fake_DoSomeStuff is going to get called for each call to DoSomeStuff in FunctionUnderTest. It's function prototype matches DoSomeStuff with the addition of a counter which CMock adds to the end, allowing us to counter the number of times we call it.

If there are more than one function being called in the loop, we can use a stub for each.

Does that make sense?

[sandeepzac]

Hello Mark,

Apologies for the delayed response.

Thank you for the detailed explanation and taking time out to explain it with an example program. It really helped me comprehend the concepts and your explanation.

The stub did really work and I was able to hit the condition, without any memory errors :)

Thanks much!

Sandeep.

[cy18]

Function resetTest() will call tearDown and setUp, so it may not be suitable in some cases. Define and call the following function ClearMockMemory works for me.

static void ClearMockMemory(void)
{
    mock_xxx_Verify();
    mock_xxx_Destroy();
    mock_xxx_Init();
    mock_yyy_Verify();
    mock_yyy_Destroy();
    mock_yyy_Init();
}

Maybe it could be further simplified by providing a function mock_xxx_ReleaseRam() ?

[sam-phillips-escrypt]

Hi, I'm having trouble including resetTest() in my test suite. I've tried forward declaring it, but is there anything else I have to include? I can't find resetTest() defined anywhere, even looking through the temporary files in my build directory.

[mvandervoord]

resetTest is automatically generated by the test runner generator. If you're manually building your own runners, then you can create your own resetTest as well. It looks like this:

void resetTest(void)
{
    CMock_Verify();
    CMock_Destroy();
    tearDown();
    setUp();
    CMock_Init();
}

Those individual functions are also autogenerated, and include the corresponding call to EACH of the Mock's verify, destroy, and init functions.

[kurt]

If you make the memory allocation dynamic in the project.yml then this will probably solve your problem. It would be nice if it didn't require this much memory though

:defines:
  # in order to add common defines:
  #  1) remove the trailing [] from the :common: section
  #  2) add entries to the :common: section (e.g. :test: has TEST defined)
  :commmon: &common_defines
    - CMOCK_MEM_DYNAMIC