How could I bypass the uncovered false branch of assert?

[musnows]

Hi, I would like to know how could I bypass the uncovered false branch of assert?

I have checked the issues related to assert and it seems that the -DNDEBUG option can be added to the g++ compile options for branch coverage testing. However, I am not familiar with this option and have collected very little information. May I ask if this will have any other impact?

It seems that after adding this compile option, the program will directly skip all assert.

I'm using lcov 2.0 and g++/gcov (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0

[henry2cox]

Adding -DNDEBUG to your compiler command line will remove the body of the assert macro - as you mention. (See the 'assert' definition in /usr/include/assert.h to see how/why.) Depending on how your code is written, this will also remove other debug/check code that is guarded by #ifndef NDEBUG ... Your tests may break if your 'debug' code has side effects. (Should not happen - and should be fixed if it does happen.)

The other thing you can do is to use the --omit-lines option to remove lines that match your assert regexp. This enables you to continue to run your check build - but to ignore/remove the assert branches. Of the two: very likely the first is the better solution.

[musnows]

Adding -DNDEBUG to your compiler command line will remove the body of the assert macro - as you mention. (See the 'assert' definition in /usr/include/assert.h to see how/why.) Depending on how your code is written, this will also remove other debug/check code that is guarded by #ifndef NDEBUG ... Your tests may break if your 'debug' code has side effects. (Should not happen - and should be fixed if it does happen.)

The other thing you can do is to use the --omit-lines option to remove lines that match your assert regexp. This enables you to continue to run your check build - but to ignore/remove the assert branches. Of the two: very likely the first is the better solution.

Thanks for your help! I will checkout the definition.

[musnows]

Hi, I found out that there is a EXPECT_DEATH in google gtest, which allows us to test if the assert is false, and the program will not terminate as a result.

#include <gtest/gtest.h>
#include <gtest/gtest-death-test.h>
#include <iostream>

void test_assert_func(int id)
{
    assert(id>0);
    std::cout << "id is good!" << std::endl;
}

TEST(testassert,testassertfailed)
{
    EXPECT_NO_THROW(test_assert_func(10));
    EXPECT_DEATH(test_assert_func(-1),"");
}

int main(int argc, char **argv)
{
    ::testing::InitGoogleTest(&argc, argv);
    return RUN_ALL_TESTS();
}

But the result shows that the false branch of assert still remain uncovered.

     337                 :           1 : void test_assert_func(int id)
     338                 :             : {
     339         [ -  + ]:           1 :     assert(id>0);
     340                 :           1 :     cout << "id is good!" << endl;
     341                 :           1 : }

gcov file shows like this 👇

function _Z16test_assert_funci called 1 returned 100% blocks executed 80%
        1:  337:void test_assert_func(int id)
        -:  338:{
       1*:  339:    assert(id>0);
branch  0 taken 0 (fallthrough)
branch  1 taken 1
call    2 never executed
        1:  340:    cout << "id is good!" << endl;
call    0 returned 1
call    1 returned 1
        1:  341:}

Is there a way for LCOV to support this situation? Or is it that gcov itself does not support this type of testing method, so branches with false assert are still not covered?

[henry2cox]

I am not sure what 'EXPECT_DEATH' does - but you should probably read up on it, to see. There are two things to keep in mind with respect to gcov and lcov:

• lcov is very, very simple. All it is doing is reading the data generated by gcov and presenting it in some slightly different way. As a result: lcov isn't going to show you anything that gcov cannot record.
• gcov emits its collected coverage data in an atexit callback. If your process does not exit normally: then no atexit callbacks are called - and no coverage data will be written. I very strongly suspect that this is what is happening in your example.

You might want to look at gcov_flush() - which can be used to force gcov to write coverage data. It is typically used in a long-running server process - but you might want to modify your assert macro definition a bit to call it before you abort.

The other thing to consider is that testing the false condition of an assert is a bit strange. Asserts are meant to be invariants...the false condition is illegal/is not supposed to happen - ever. It seems as if you want to test your asserts because you are using them to test error conditions - which is not their intended use. malloc() returning null is not a good thing to assert. Passing a null pointer to a method which does not support it probably is a good thing to assert (...depending on your code base and who is using it).

Lastly: it may be useful/instructive to run a very simple example which only calls your assert fail case - and see what happens. Remove any pre-exising .gcda files and run your example. What coverage data is dumped. Is any coverage data dumped?

[musnows]

I am not sure what 'EXPECT_DEATH' does - but you should probably read up on it, to see. There are two things to keep in mind with respect to gcov and lcov:

• lcov is very, very simple. All it is doing is reading the data generated by gcov and presenting it in some slightly different way. As a result: lcov isn't going to show you anything that gcov cannot record.
• gcov emits its collected coverage data in an atexit callback. If your process does not exit normally: then no atexit callbacks are called - and no coverage data will be written. I very strongly suspect that this is what is happening in your example.

You might want to look at gcov_flush() - which can be used to force gcov to write coverage data. It is typically used in a long-running server process - but you might want to modify your assert macro definition a bit to call it before you abort.

The other thing to consider is that testing the false condition of an assert is a bit strange. Asserts are meant to be invariants...the false condition is illegal/is not supposed to happen - ever. It seems as if you want to test your asserts because you are using them to test error conditions - which is not their intended use. malloc() returning null is not a good thing to assert. Passing a null pointer to a method which does not support it probably is a good thing to assert (...depending on your code base and who is using it).

Lastly: it may be useful/instructive to run a very simple example which only calls your assert fail case - and see what happens. Remove any pre-exising .gcda files and run your example. What coverage data is dumped. Is any coverage data dumped?

Thanks for your fast reply! 🫡

In fact, I only did this to achieve an internal standard of branch coverage. It seems that the false branch of assert may not be able to successfully handle it, so I have decided to use -DNDEBUG and remove EXPECT_DEATH testing method for assset.

I also know that testing the false branch of an assert is indeed unreasonable and meaningless, but the internal standard expects us to achieve the highest possible branch coverage, so I am trying every way to cover this branch.

[GuWei007]

Hi, I found out that there is a EXPECT_DEATH in google gtest, which allows us to test if the assert is false, and the program will not terminate as a result.

#include <gtest/gtest.h>
#include <gtest/gtest-death-test.h>
#include <iostream>

void test_assert_func(int id)
{
    assert(id>0);
    std::cout << "id is good!" << std::endl;
}

TEST(testassert,testassertfailed)
{
    EXPECT_NO_THROW(test_assert_func(10));
    EXPECT_DEATH(test_assert_func(-1),"");
}

int main(int argc, char **argv)
{
    ::testing::InitGoogleTest(&argc, argv);
    return RUN_ALL_TESTS();
}

But the result shows that the false branch of assert still remain uncovered.

     337                 :           1 : void test_assert_func(int id)
     338                 :             : {
     339         [ -  + ]:           1 :     assert(id>0);
     340                 :           1 :     cout << "id is good!" << endl;
     341                 :           1 : }

gcov file shows like this 👇

function _Z16test_assert_funci called 1 returned 100% blocks executed 80%
        1:  337:void test_assert_func(int id)
        -:  338:{
       1*:  339:    assert(id>0);
branch  0 taken 0 (fallthrough)
branch  1 taken 1
call    2 never executed
        1:  340:    cout << "id is good!" << endl;
call    0 returned 1
call    1 returned 1
        1:  341:}

Is there a way for LCOV to support this situation? Or is it that gcov itself does not support this type of testing method, so branches with false assert are still not covered?

What tool do you use to see specific branch coverage?

function _Z16test_assert_funci called 1 returned 100% blocks executed 80% 1: 337:void test_assert_func(int id) -: 338:{ 1*: 339: assert(id>0); branch 0 taken 0 (fallthrough) branch 1 taken 1 call 2 never executed 1: 340: cout << "id is good!" << endl; call 0 returned 1 call 1 returned 1 1: 341:}

[musnows]

Hi, I found out that there is a EXPECT_DEATH in google gtest, which allows us to test if the assert is false, and the program will not terminate as a result.

#include <gtest/gtest.h>
#include <gtest/gtest-death-test.h>
#include <iostream>

void test_assert_func(int id)
{
    assert(id>0);
    std::cout << "id is good!" << std::endl;
}

TEST(testassert,testassertfailed)
{
    EXPECT_NO_THROW(test_assert_func(10));
    EXPECT_DEATH(test_assert_func(-1),"");
}

int main(int argc, char **argv)
{
    ::testing::InitGoogleTest(&argc, argv);
    return RUN_ALL_TESTS();
}

But the result shows that the false branch of assert still remain uncovered.

     337                 :           1 : void test_assert_func(int id)
     338                 :             : {
     339         [ -  + ]:           1 :     assert(id>0);
     340                 :           1 :     cout << "id is good!" << endl;
     341                 :           1 : }

gcov file shows like this 👇

function _Z16test_assert_funci called 1 returned 100% blocks executed 80%
        1:  337:void test_assert_func(int id)
        -:  338:{
       1*:  339:    assert(id>0);
branch  0 taken 0 (fallthrough)
branch  1 taken 1
call    2 never executed
        1:  340:    cout << "id is good!" << endl;
call    0 returned 1
call    1 returned 1
        1:  341:}

Is there a way for LCOV to support this situation? Or is it that gcov itself does not support this type of testing method, so branches with false assert are still not covered?

What tool do you use to see specific branch coverage?

function _Z16test_assert_funci called 1 returned 100% blocks executed 80% 1: 337:void test_assert_func(int id) -: 338:{ 1*: 339: assert(id>0); branch 0 taken 0 (fallthrough) branch 1 taken 1 call 2 never executed 1: 340: cout << "id is good!" << endl; call 0 returned 1 call 1 returned 1 1: 341:}

gcov in ubuntu

gcov  (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0

command in my demo (I cannot guarantee that this command will be correct, but currently it is working properly)

g++ -std=c++17 test.cpp -o test -lgtest -lgtest_main -pthread -fprofile-arcs -ftest-coverage -fprofile-update=atomic
./test
gcov -b -c -o . test.cpp

gcov will generate test.cpp.gcov files, which includes the reports you quoted;

[henry2cox]

gcov will generate test.cpp.gcov files, which includes the reports you quoted;

Yep. And this is exactly the data that lcov reads in the lcov --capture ... step.

As mentioned above: lcov is an extremely simple tool. It relies on the compiler/toolchain to generate the data - and merely turns that into something more usable.