Snaipe / Criterion

A cross-platform C and C++ unit testing framework for the 21st century
MIT License
1.95k stars 177 forks source link

Testing with Valgrind #530

Open EmilChoparinov opened 1 month ago

EmilChoparinov commented 1 month ago

In issue #21 it was stated to use the --no-early-exit flag to run get a clean valgrind run but since then the flag is deprecated. What's the replacement? I would really love to use this library with valgrind.

Code:

#include <criterion/criterion.h>
Test(Hello,World)
{

}

Valgrind run:

emil@DESKTOP-DDFEUUB:~/GECS$ ./tests/bin/nmap_tests -l
Hello: 1 test
└── World
emil@DESKTOP-DDFEUUB:~/GECS$ valgrind --leak-check=full --trace-children=yes  tests/bin/nmap_tests --no-early-exit
==19649== Memcheck, a memory error detector
==19649== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==19649== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==19649== Command: tests/bin/nmap_tests --no-early-exit
==19649== 
--no-early-exit is now deprecated as it no longer does anything.
==19654== Memcheck, a memory error detector
==19654== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==19654== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==19654== Command: /home/emil/GECS/tests/bin/nmap_tests
==19654== 
==19654== 
==19654== HEAP SUMMARY:
==19654==     in use at exit: 416,468 bytes in 1,467 blocks
==19654==   total heap usage: 3,283 allocs, 1,816 frees, 860,393 bytes allocated
==19654== 
==19654== LEAK SUMMARY:
==19654==    definitely lost: 0 bytes in 0 blocks
==19654==    indirectly lost: 0 bytes in 0 blocks
==19654==      possibly lost: 0 bytes in 0 blocks
==19654==    still reachable: 416,468 bytes in 1,467 blocks
==19654==         suppressed: 0 bytes in 0 blocks
==19654== Reachable blocks (those to which a pointer was found) are not shown.
==19654== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==19654== 
==19654== For lists of detected and suppressed errors, rerun with: -s
==19654== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
[====] Synthesis: Tested: 1 | Passing: 1 | Failing: 0 | Crashing: 0 
==19649== 
==19649== HEAP SUMMARY:
==19649==     in use at exit: 416,468 bytes in 1,467 blocks
==19649==   total heap usage: 3,612 allocs, 2,145 frees, 899,899 bytes allocated
==19649== 
==19649== LEAK SUMMARY:
==19649==    definitely lost: 0 bytes in 0 blocks
==19649==    indirectly lost: 0 bytes in 0 blocks
==19649==      possibly lost: 0 bytes in 0 blocks
==19649==    still reachable: 416,468 bytes in 1,467 blocks
==19649==         suppressed: 0 bytes in 0 blocks
==19649== Reachable blocks (those to which a pointer was found) are not shown.
==19649== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==19649== 
==19649== For lists of detected and suppressed errors, rerun with: -s
==19649== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Ximaz commented 1 month ago

The replacement is doing this :

CRITERION_NO_EARLY_EXIT=1 valgrind $VALGRIND_FLAGS ./unit_tests

You have to export this variable first.

EDIT : I read about the CRITERION_NO_EARLY_EXIT somewhere I can't remember. But it appears that, this env variable also doesn't do anything anymore, because it doesn't appear here : https://criterion.readthedocs.io/en/master/env.html#environment-variables