add address and undefined sanitizers to github action

mo-xiaoming commented 2 years ago

Besides being ugly, there are few things to note

Currently leak detection is disabled
Has to add something like -Og to debug flags, otherwise, get warning _FORTIFY_SOURCE requires compiling with optimization (-O). This is a long standing nix/gcc bug https://github.com/NixOS/nixpkgs/issues/18995
Default added to path, otherwise, nix generates something like hobbesPackages/gcc-8/llvm-9//hobbes. Don't know how to get ride of //, so a Default added
Adding sanitizers make build slow, and Release build with sanitizer and debug symbols takes forever to build. So there is only one active(gcc8, llvm11?) version is checked by sanitizers

mo-xiaoming commented 2 years ago

It just proved its value, a null pointer reference was detected

/nix/store/rnh1x6pijwlmk14cy5jnyl5v51mc72wi-gcc-8.4.0/include/c++/8.4.0/bits/stl_iterator.h:797:17: runtime error: reference binding to null pointer of type 'const unsigned char'
    #0 0x952bef in __gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >::operator*() const /nix/store/rnh1x6pijwlmk14cy5jnyl5v51mc72wi-gcc-8.4.0/include/c++/8.4.0/bits/stl_iterator.h:797
    #1 0x952bef in void hobbes::fregion::writes<__gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > > >(hobbes::fregion::imagefile*, __gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >) ../include/hobbes/fregion.H:265
    #2 0x952bef in hobbes::fregion::write(hobbes::fregion::imagefile*, std::vector<unsigned char, std::allocator<unsigned char> > const&) ../include/hobbes/fregion.H:273
    #3 0x952bef in hobbes::fregion::addBinding(hobbes::fregion::imagefile*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<unsigned char, std::allocator<unsigned char> > const&, unsigned long) ../include/hobbes/fregion.H:578
    #4 0x95a4f9 in hobbes::fregion::openFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, unsigned short, unsigned short, unsigned long) ../include/hobbes/fregion.H:903
    #5 0x8e118f in hobbes::fregion::writer::writer(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ../include/hobbes/fregion.H:1945
    #6 0x8e118f in makeTestData(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ../test/Python.C:98
    #7 0x8e8c34 in test_Python_FRegion() ../test/Python.C:272
    #8 0x71650d in TestCoord::runTestGroups(Args const&) ../test/Main.C:64
    #9 0x7297f6 in main ../test/Main.C:143
    #10 0x7ffff6184dec in __libc_start_main (/nix/store/v8q6nxyppy1myi3rxni2080bv8s9jxiy-glibc-2.32-40/lib/libc.so.6+0x27dec)
    #11 0x4927d9 in _start (/build/source/build/hobbes-test+0x4927d9)

WIP

smunix commented 2 years ago

Run sanitizers with clang only (not gcc)

mo-xiaoming commented 2 years ago

There are few changed in the last two commits

Previous crash was caused by accessing the first element of an empty vector https://github.com/morganstanley/hobbes/blob/30574be2455852c3d9d98d1e554873e61481d7b7/include/hobbes/fregion.H#L903

test/Python.C failed on my ubuntu20.04

    Python (1 test)
  ---------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/hdb-unittest-1.py", line 14, in <module>
    f = fregion.FRegion(sys.argv[1])
  File "/home/ubuntu/hobbes-llvm11/scripts/fregion.py", line 1068, in __init__
    bind.reader = makeReader({}, bind.ty)
  File "/home/ubuntu/hobbes-llvm11/scripts/fregion.py", line 1030, in makeReader
    return TyCase(readerDisp).apply(ty)
  File "/home/ubuntu/hobbes-llvm11/scripts/fregion.py", line 247, in apply
    return self.dtors["prim"](ty)
  File "/home/ubuntu/hobbes-llvm11/scripts/fregion.py", line 1019, in <lambda>
    "prim":    lambda p:  makePrimReader(renv, p),
  File "/home/ubuntu/hobbes-llvm11/scripts/fregion.py", line 958, in makePrimReader
    raise Exception("I don't know how to decode the primitive type: " + p.name)
Exception: I don't know how to decode the primitive type: b'bool'
    FRegion FAIL (317.155ms)

include(FindPythonInterp) finds python3 on my system, but fregion.py only can handle python2 strings (bytes by default), and python3 uses unicode as default, so we can see b'bool' doesn't match 'bool'

Migrating this script to be python3 compatible is just too much work, so I just simply force it to search python2.7

enableDebugging doesn't seem to change CMAKE_BUILD_TYPE, it still use Release, which doesn't set proper flags

diff --git a/nix/overlays.nix b/nix/overlays.nix
index 14b1944..55f7636 100644
--- a/nix/overlays.nix
+++ b/nix/overlays.nix
@@ -79,7 +79,6 @@ let
       inherit version src meta doCheck doTarget;

       dontStrip = true;
-      cmakeBuildType="Debug";
       cmakeFlags = [
         "-DUSE_ASAN_AND_UBSAN:BOOL=ON"
       ];
@@ -119,7 +118,7 @@ in {
       (llvmVersion: {
         name = "clang-" + toString llvmVersion + "-ASanAndUBSan";
         value = recurseIntoAttrs ({
-          hobbes = dbg (callPackage withCLANGAsanAndUBSan { inherit llvmVersion; });
+          hobbes = enableDebugging (callPackage withCLANGAsanAndUBSan { inherit llvmVersion; });
         });
       }) llvmVersions));
 }

Now CI failed on an function pointer conversion which I've seen at least a month ago, then I've not been able to reproduce it since then...

WIP

morganstanley / hobbes

add address and undefined sanitizers to github action #417