MultiThreading for MockSupport

[mkeeler]

Should CppUMock and the Memory Leak Detector plugin support multithreading. I have a case where I have compile a shared library with three methods. Each of these 3 calls mock().actualCall and mock().intReturnValue. This shared library is then loaded by some of the code that I am trying to test (I am testing that my plugin loading code calls the right functions etc..). In this case one of several things happens. Sometimes there is a segfault, sometimes it gets aborted in glibc due to a double free/corruption error, and sometimes the Memory leak detector complains about leaks in the mocking code. Also it sometimes errors saying that calls I was expecting didn't happen when I can see that they actually did.

When there is a double free problem the stack trace includes the following calls to mocking facilities. _ZN16MockFunctionCallD0Ev+0x9)[0x42e599] _ZN11MockSupport10actualCallERK12SimpleString+0x3a)[0x42b12a]

Overall it appears as though CppUMock and the memory leak detector are not thread safe and I was wondering if that suspicion could be confirmed.

[mkeeler]

I just tested something else.

Originally I had two threads being created one after the other. Each one would call a different function in my mock plugin. These threads were being started simultaneously. I inserted a 10000 microsecond sleep between starting the two threads and now it reliably works.

It would appear that the underlying mocking code isn't synchronized in any way and thus makes it unsuitable for testing multithreaded programs.

[basvodde]

@mkeeler There is a pull request open related to the memory leak detector. Related to mocking, yup, I don't think it is thread-safe. That might be something we could look at in the future. I'll keep this issue open. Thanks!

[mys007]

One issue with multi-threading and mocking is the fact that calls are checked on the fly and not only at the end by MockSupport::checkExpectations(). Now if an ActualCall happens to run in a different thread from the test-originator's and fails, this propagates to MockFailureReporterTestTerminator, which either throws an exception (crash; application typically doesn't handle that) or performs longjmp (crash; jump between threads). The hot-fix is to disable calling NormalTestTerminator::exitCurrentTest() in MockFailureReporterTestTerminator.

[jwgrenning]

CppUTest is note designed to be used with different threads helping to meet expectations. For unit testing with CppUTest you should use single threads. This probably means organizing your code to separate the concerns of what is done by a thread and its concurrency issues.

If you are going to have multiple thread (not that you would generally need that for unit testing) the thread setting expectations needs to be the thread checking them all. All threads must be joined before the end of a test.

Here is an three article that show a unit test examples in a concurrent execution environment. Its not using CppUMock, but it shows how you might stub out concurrency in unit tests.

http://www.renaissancesoftware.net/blog/?s=rtos&submit=Search

James

---

James Grenning Author of TDD for Embedded C www.wingman-sw.com http://pragprog.com/titles/jgade/ www.wingman-sw.com/blog www.twitter.com/jwgrenning

On 18 Jun 2014, at 4:35, mys007 wrote:

One issue with multi-threading and mocking is the fact that calls are checked on the fly and not only at the end by MockSupport::checkExpectations(). Now if an ActualCall happens to run in a different thread from the test-originator's and fails, this propagates to MockFailureReporterTestTerminator, which either throws an exception (crash; application typically doesn't handle that) or performs longjmp (crash; jump between threads). The hot-fix is to disable calling NormalTestTerminator::exitCurrentTest() in MockFailureReporterTestTerminator.

---

Reply to this email directly or view it on GitHub: https://github.com/cpputest/cpputest/issues/234#issuecomment-46414589

[basvodde]

@mys007 Disabling will prevent a crash, but it probably won't lead to the correct behavior. If we really want to make the mocking threadsafe, then I suspect there is more to it than not letting it crash.

The crash you mentioned would also happen with 'normal' CppUTest as any assert will throw an exception.

[mys007]

@basvodde Well, depends on the definition of correct behaviour. If it means that the test fails at the right place and this is reported, then it's IMHO still valid (I also disabled the functionality in MockCheckedActualCall::checkStateConsistency). The crash shouldn't happen with the normal CppUTest, as the call of the test is wrapped by handlers in Utest::run(), so there is a handler during stack-unwinding. However, when a new thread is started, it has its own stack and the main thread function is not wrapped in anything.

[basvodde]

It won't be the correct behavior as it probably won't fail the test (and it will probably crash randomly as the access to the Mock datastructures are not protected.

In normal CppuTest the same thing would happen if you assert from a different thread then that the test is run from. In that case, it will also do a throw which ends at the end of the thread, same situation.

But, I'm still leaving this issue open to check more carefully if we can make CppUTest asserts and mocking thread-friendly.

[hpatel16]

Hello Bas,

I wanted to check if there is any update on making CppUMock thread safe ? Is there any activity in this area ? Do you have any design ideas or suggestions around how to make this thread safe ?

We use CppUtest for Unit Test and are looking to start using it for Integration Test in a multi-threaded environment and I see this issue as a limiting factor to use this for IT.

thanks, Hetal Patel.

[basvodde]

No, there is no activity on this. IMHO making the mocking truly multi-threading is much harder as it will require cross-thread coordination. (e.g. if the thread has a failing mock, then it will need to give control to the main thread).

IMHO this would be a huge work.