libcheck / check

A unit testing framework for C
GNU Lesser General Public License v2.1
1.07k stars 210 forks source link

Add ck_skip() and ck_skip_msg(...) macros [WIP] [DISCUSS] #263

Closed jbboehr closed 4 years ago

jbboehr commented 4 years ago

These are all analogues of ck_abort(), ck_abort_msg(), and _ck_assert_failed()

Since they basically work like ck_abort(), I decided to use a flag wasskipped in FailMsg instead of adding a new message type.

As it is, I'm not including skipped tests in the success percentage, but they are included in total tests run. I considered adding srunner_ntests_skipped() and srunner_skipped() but decided to hold off since they are included in srunner_results()

In tr_type_str, I assigned skip to K because the default return value was S already. I'm not sure if there's a more appropriate value here.

I'm personally using check with the autotools tap driver, and it seems to be working alright so far.

Let me know if you have any thoughts and, of course, feel free to request changes.

Closes #178

thom311 commented 4 months ago

I think a skip should not abort the test, unlike an assertion failure. Otherwise, if you run without fork-mode, an earlier skip will prevent all followup tests from running. With assertion failures that is fine, because you better debug the first failure you encounter. For skip, that seems undesirable.

Compare to glib's g_test_skip(), which just remembers the verdict of the test, and evaluates the test result at the end.

Yes, it means, usually the user would follow up that with an additional return, like

if (!has_dependency()) {
     ck_skip("Lacks dependency");
     return;
}

I think this would be a good feature.