Adding TEST_ASSERT (like TEST_CHECK but aborting the test on a failed condition)

[AndreRenaud]

TEST_CHECK currently returns a testable value. However for most of the tests that I'm writing I essentially use TEST_CHECK as an assert (ie: if it fails, there isn't much point continuing with the test). Is there any value in adding a TEST_ASSERT macro which behaves this way? Basically: #define TEST_ASSERT(a) do { if (!TEST_CHECK(a)) return; } while (0)

[mity]

Is there any value in adding a TEST_ASSERT macro which behaves this way?

For example some stuff like this:

SomeStruct* s = allocate_some_struct();
if(TEST_CHECK(s != NULL)) {
    // more checks here; e.g. that the struct has some properly initialized members.
    // (they would have no sense to do if the struct is not allocated)
}

#define TEST_ASSERT(a) do { if (!TEST_CHECK(a)) return; } while (0)

I agree that the assert-like approach can be handy sometimes. But consider also some more complicated tests where the main test function may call some helper functions and those may also do some checks too. See e.g. https://github.com/mity/centijson/blob/master/tests/test-json.c

Or, if the test function iterates over many test vectors, it is imho better to check all the vectors and not to stop on the 1st failure. Such an example: https://github.com/hsluv/hsluv-c/blob/master/tests/test_hsluv.c

[AndreRenaud]

I'm not advocating for removing TEST_CHECK, or its use in that manor, I was suggesting adding something in addition to that. Sometimes you have a vector of test cases, but frequently you only have one. I find myself using TEST_CHECK, but then really only caring about the first thing it fails on and just ignoring everything after that.

[mity]

I see. But I cannot see how such assert could be implemented to really work reliably everywhere.

Consider user can use --no-exec or be on a platform where we run everything in a single process. So we cannot use exit(), abort() or anything like that.

Your macro #define TEST_ASSERT(a) do { if (!TEST_CHECK(a)) return; } while (0) imho does not really behave as assertion (when used in a called helper function) so it would likely be a source of confusion.

In C++, we could maybe throw an exception. But what in C?

The only other option coming to my mind is setjmp() + longjmp(). But yet again, it is not portable. So what about platforms which do not have it?

[AndreRenaud]

Yes, you're right, I had missed the usage of it in a helper function. I'll give it some more thought and see if I can come up with a suitable C proposal.

[mity]

According to https://en.cppreference.com/w/c/program/setjmp, <setjmp.h> is provided by the standard C lib so it should be more or less portable. Hence, I've added TEST_ASSERT and TEST_ASSERT_.

They have the same interface as TEST_CHECK and TEST_CHECK_ except they abort when the check fails, either by abort() or by longjmp(), depending whether the test is being executed in the child process or in the main one.

I would see it as an experimental feature for now: If it turns out it causes too many problems, I will revoke it again.

(Reopening until this is made clear.)

[AndreRenaud]

Thanks for coming back to this. I had a quick look - does it need to be conditional for test_abort - ie: https://github.com/mity/acutest/pull/27? I may be misunderstanding how you intended it to be used.

[gdamore]

FYI, I find TEST_ASSERT() to be useful -- I'm using it only in cases where abort() is the right thing to do -- but essentially it would be bad for tests to continue if they fail during initial setup.

[mity]

Good to know.

[mity]

Closing. I've found it useful too. It does not seem to cause me problems e.g. with respect to debuggers or anything.