ThrowTheSwitch / Ceedling

Ruby-based unit testing and build system for C projects
http://throwtheswitch.org
Other
597 stars 246 forks source link

Adress sanitizer errors shows up as crash (no further info) on pre-release 1.0.0-3d9cd04 #924

Open pileghoff opened 2 months ago

pileghoff commented 2 months ago

Testing pre-release 1.0.0-3d9cd04, using aderss sanitizer, setup as:

:flags:
  :test:
    :compile:
      - -fsanitize=address
    :link:
      - -fsanitize=address

When i generate a test case that fails due to a memory leak:

#include <stdlib.h>

void test_mem_leak() {
    int* test = malloc(sizeof(int));
}

I see:

👟 Executing
------------
Running test_mem_leak.out...
☠️ ERROR: Test executable `test_mem_leak.out` seems to have crashed

Running Test Suite Reports
--------------------------
Generating artifact .build/artifacts/test/report_junit.xml...

-------------------
FAILED TEST SUMMARY
-------------------
[test/test_mem_leak.c]
  Test: test_mem_leak
  At line (3): "Test executable crashed"

-----------------------
❌ OVERALL TEST SUMMARY
-----------------------
TESTED:  1
PASSED:  0
FAILED:  1
IGNORED: 0

---------------------
BUILD FAILURE SUMMARY
---------------------
Unit test failures.

I expected the following info from asan to also be included somewhere:

==1==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 4 byte(s) in 1 object(s) allocated from:
    #0 0x7f7ecdb4e887 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x55a35118d39e in test_mem_leak test/test_mem_leak.c:4
    #2 0x55a35118d555 in run_test .build/test/runners/test_mem_leak_runner.c:68
    #3 0x55a35118d5b8 in main .build/test/runners/test_mem_leak_runner.c:84
    #4 0x7f7ecd89ad8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

SUMMARY: AddressSanitizer: 4 byte(s) leaked in 1 allocation(s).
mvandervoord commented 2 months ago

Hi. We don't currently have an asan plugin (though I'm thinking we should?). Without that plugin, the extra output is just noise and is therefore subject to the verbosity level you're running at.

If you try the following, do you see the details?

ceedling -v=obnoxious test:mem_leak

If so, that can at least help you for now. If not, I'd like to help figure out where your output is going.

Either way, I'd appreciate it if you either left this issue open, or added a followup issue with the suggestion that we write a plugin to support ASan. I suspect we're not alone in thinking this would be very useful.

pileghoff commented 2 months ago

As expected, adding -v=obnoxious produces the wanted output (along with a lot of unwanted of course). While a ASan plugin would be nice and solve this specific issue, wouldn't it make sense to print out stderr when a test case crashes? (maybe truncated?) This way, you could help out with all other potential crashes.

mvandervoord commented 2 months ago

That would definitely make sense for when people are building native executables. So we could default to having this behavior, but possibly disable it if they're using a simulator or other test rig. Good thought.

pileghoff commented 2 months ago

We have build a small plugin internally that enables asan and prints out the report as expected. It has no fancy options or anything, but i will clean it up and make the PR in the coming days.