lcompilers / lpython

Python compiler
https://lpython.org/
Other
1.5k stars 158 forks source link

[CI] add sanititizer to detect undefined behavior #2547

Open syheliel opened 7 months ago

syheliel commented 7 months ago

When walking around the recent bug issue, I try to build the binary with sanitizer, and I find there are some undefined behavior and memory leak in the code, such as:

> ./src/bin/lpython  ./tests/expr1.py
/home/xxx/lpython/src/lpython/python_ast.h:7728:13: runtime error: load of value 2, which is not a valid value for type 'bool'
/home/xxx/lpython/src/lpython/python_ast.h:7420:13: runtime error: load of value 2, which is not a valid value for type 'bool'
/home/xxx/lpython/src/lpython/python_ast.h:7388:13: runtime error: load of value 2, which is not a valid value for type 'bool' 

...
Indirect leak of 25 byte(s) in 1 object(s) allocated from:
    #0 0x151d8d6aff67 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:99
    #1 0x56483ddc56ad in llvm::TargetMachine::TargetMachine(llvm::Target const&, llvm::StringRef, llvm::Triple const&, llvm::StringRef, llvm::StringRef, llvm::TargetOptions const&) (/home/zzh/lpython/src/bin/lpython+0x154556ad)
    #2 0x100000000  (<unknown module>)

Indirect leak of 24 byte(s) in 1 object(s) allocated from:
    #0 0x151d8d6aff67 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:99
    #1 0x56483d53f3c6 in llvm::X86TargetMachine::getSubtargetImpl(llvm::Function const&) const (/home/zzh/lpython/src/bin/lpython+0x14bcf3c6)

Indirect leak of 24 byte(s) in 1 object(s) allocated from:
    #0 0x151d8d6aff67 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:99
    #1 0x56483ddc353e in llvm::TargetLoweringObjectFile::Initialize(llvm::MCContext&, llvm::TargetMachine const&) (/home/zzh/lpython/src/bin/lpython+0x1545353e)

Indirect leak of 23 byte(s) in 1 object(s) allocated from:
    #0 0x151d8d6aff67 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:99
    #1 0x56483e7db39e in llvm::ValueSymbolTable::createValueName(llvm::StringRef, llvm::Value*) (/home/zzh/lpython/src/bin/lpython+0x15e6b39e)

Indirect leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x151d8d6aff67 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:99
    #1 0x56483d5381d4 in llvm::X86Subtarget::X86Subtarget(llvm::Triple const&, llvm::StringRef, llvm::StringRef, llvm::X86TargetMachine const&, llvm::MaybeAlign, unsigned int, unsigned int) (/home/zzh/lpython/src/bin/lpython+0x14bc81d4)

SUMMARY: AddressSanitizer: 318134 byte(s) leaked in 392 allocation(s).

You can modify the build1.sh script to add sanitizer into the binary the check the problem:

# build1.sh
cmake \
    -DCMAKE_BUILD_TYPE=Debug \
    -DCMAKE_EXPORT_COMPILE_COMMANDS=1 \
    -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
    -DWITH_LLVM=yes \
    -DLPYTHON_BUILD_ALL=yes \
    -DWITH_STACKTRACE=yes \
    -DWITH_RUNTIME_STACKTRACE=yes \
    -DWITH_LSP=no \
    -DWITH_LFORTRAN_BINARY_MODFILES=no \
    -DCMAKE_CXX_FLAGS="-fsanitize=address,undefined" \
    -DCMAKE_PREFIX_PATH="$CMAKE_PREFIX_PATH_LPYTHON;$CONDA_PREFIX" \
    -DCMAKE_INSTALL_PREFIX=`pwd`/inst \
    .

It seems that we haven't considered add sanitizer into our CI testing. So existing codebase will give an memory leak on even simple a+b problem. I think it may be related to #2498 and #2544