search

bitcoin / bitcoin

Bitcoin Core integration/staging tree
https://bitcoincore.org/en/download
MIT License
78.42k stars 36.17k forks source link

ci: fuzz_msan failed with ==4201==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x55f0c9bdeffb in SetArgs #30760

Closed maflcko closed 1 week ago

maflcko commented 2 weeks ago

I recall a similar issue previously.

+ [[ x86_64-pc-linux-gnu = *-mingw32 ]]
+ '[' -n '' ']'
+ '[' false = true ']'
+ '[' '' = true ']'
+ '[' false = true ']'
+ '[' false = true ']'
+ '[' true = true ']'
+ LD_LIBRARY_PATH=/ci_container_base/depends/x86_64-pc-linux-gnu/lib
+ test/fuzz/test_runner.py -j16 -l DEBUG /qa_assets/fuzz_seed_corpus/ --empty_min_time=60
==4201==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x55f0c9bdeffb in SetArgs(int, char**) ci/scratch/build-x86_64-pc-linux-gnu/src/test/fuzz/util/./src/test/fuzz/fuzz.cpp:50:5
    #1 0x55f0c9bdeffb in LLVMFuzzerInitialize ci/scratch/build-x86_64-pc-linux-gnu/src/test/fuzz/util/./src/test/fuzz/fuzz.cpp:216:5
    #2 0x55f0c8f48508 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /msan/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:650:5
    #3 0x55f0c8f758b2 in main /msan/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    #4 0x7f7e0b6be1c9  (/lib/x86_64-linux-gnu/libc.so.6+0x2a1c9) (BuildId: 6d64b17fbac799e68da7ebd9985ddf9b5cb375e6)
    #5 0x7f7e0b6be28a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a28a) (BuildId: 6d64b17fbac799e68da7ebd9985ddf9b5cb375e6)
    #6 0x55f0c8f3cbb4 in _start (/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/src/test/fuzz/fuzz+0x883bb4)

  Member fields were destroyed
    #0 0x55f0c900849d in __sanitizer_dtor_callback_fields /msan/llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:1048:5
    #1 0x55f0c8f3ba72 in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::~basic_string() /msan/cxx_build/include/c++/v1/string:840:44
    #2 0x55f0c8f3ba72 in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>::~basic_string() /msan/cxx_build/include/c++/v1/string:1106:3
    #3 0x55f0c8f3ba72 in std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, BCLog::LogFlags>::~pair() /msan/cxx_build/include/c++/v1/__utility/pair.h:80:29
    #4 0x55f0c8f3ba72 in __cxx_global_var_init ci/scratch/build-x86_64-pc-linux-gnu/src/util/./src/logging.cpp:170:66
    #5 0x55f0c8f3ba72 in _GLOBAL__sub_I_logging.cpp ci/scratch/build-x86_64-pc-linux-gnu/src/util/./src/logging.cpp
    #6 0x7f7e0b6be303 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a303) (BuildId: 6d64b17fbac799e68da7ebd9985ddf9b5cb375e6)
    #7 0x55f0c8f3cbb4 in _start (/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/src/test/fuzz/fuzz+0x883bb4)

SUMMARY: MemorySanitizer: use-of-uninitialized-value ci/scratch/build-x86_64-pc-linux-gnu/src/test/fuzz/util/./src/test/fuzz/fuzz.cpp:50:5 in SetArgs(int, char**)
Exiting
Traceback (most recent call last):
  File "/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 411, in <module>
    main()
  File "/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 115, in main
    test_list_all = parse_test_list(
                    ^^^^^^^^^^^^^^^^
  File "/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/test/fuzz/test_runner.py", line 397, in parse_test_list
    test_list_all = subprocess.run(
                    ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/src/test/fuzz/fuzz' returned non-zero exit status 1.

9730288a0cd3f33021ef00fb2d95e5216d10ab61 is the first bad commit
commit 9730288a0cd3f33021ef00fb2d95e5216d10ab61
Date:   Wed Jul 24 11:54:41 2024 +0100

    ci: Migrate CI scripts to CMake
vasild commented 2 weeks ago 
    #0 0x55f0c9bdeffb in SetArgs(int, char**) ci/scratch/build-x86_64-pc-linux-gnu/src/test/fuzz/util/./src/test/fuzz/fuzz.cpp:50:5

https://github.com/bitcoin/bitcoin/blob/3ee1521c048a41bc1e876d90261db677ec958f72/src/test/fuzz/fuzz.cpp#L49-L50

So argc is uninitialized? It comes from here:

    #1 0x55f0c9bdeffb in LLVMFuzzerInitialize ci/scratch/build-x86_64-pc-linux-gnu/src/test/fuzz/util/./src/test/fuzz/fuzz.cpp:216:5

https://github.com/bitcoin/bitcoin/blob/3ee1521c048a41bc1e876d90261db677ec958f72/src/test/fuzz/fuzz.cpp#L214-L216

    #2 0x55f0c8f48508 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /msan/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:650:5

https://github.com/llvm/llvm-project/blob/main/compiler-rt/lib/fuzzer/FuzzerDriver.cpp#L645-L651

    #3 0x55f0c8f758b2 in main /msan/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10

https://github.com/llvm/llvm-project/blob/main/compiler-rt/lib/fuzzer/FuzzerMain.cpp#L19-L20

Looks bogus - how could argc be uninitialized in int main(int argc, ...)!?

Was it compiled in a different way with cmake compared to autotools?

Also this is puzzling:

  Member fields were destroyed
...
    #4 0x55f0c8f3ba72 in __cxx_global_var_init ci/scratch/build-x86_64-pc-linux-gnu/src/util/./src/logging.cpp:170:66

https://github.com/bitcoin/bitcoin/blob/3ee1521c048a41bc1e876d90261db677ec958f72/src/logging.cpp#L170-L172

What has LOG_CATEGORIES_BY_STR to do with argc being uninitialized?

maflcko commented 2 weeks ago

The question is why does it happen when fuzz is compiled with cmake, but not when compiled with autotools?

The report itself is likely a false positive.

maflcko commented 2 weeks ago

Let me check https://github.com/bitcoin/bitcoin/pull/29837/files

maflcko commented 2 weeks ago

Ref https://github.com/bitcoin/bitcoin/issues/28570

fanquake commented 2 weeks ago

Looking at the output of an msan fuzz ci run from qa-assets: https://cirrus-ci.com/task/5304183451025408?logs=ci#L797, it looks like the C++ compiler flags line is just missing from the CMake output?:

Cross compiling ....................... FALSE
C++ compiler .......................... Clang 18.1.3, /usr/bin/clang++
Preprocessor defined macros ........... ABORT_ON_FAILED_ASSUME
Linker flags .......................... -fsanitize=memory -fsanitize-memory-track-origins=2 -fno-omit-frame-pointer -g -O1 -fno-optimize-sibling-calls -nostdinc++ -nostdlib++ -isystem /msan/cxx_build/include/c++/v1 -L/msan/cxx_build/lib -Wl,-rpath,/msan/cxx_build/lib -lc++ -lc++abi -lpthread -Wno-unused-command-line-argument -O2 -g -fsanitize=fuzzer,memory -fstack-protector-all -fcf-protection=full -fstack-clash-protection -Wl,-z,relro -Wl,-z,now -Wl,-z,separate-code -fPIE -pie

Not sure why that would happen, but I'm assuming that is related.

maflcko commented 2 weeks ago

Yeah, an alternative to get them may be --verbose. I'll try that next week.

hebasto commented 2 weeks ago

Looking at the output of an msan fuzz ci run from qa-assets: https://cirrus-ci.com/task/5304183451025408?logs=ci#L797, it looks like the C++ compiler flags line is just missing from the CMake output?:

Cross compiling ....................... FALSE
C++ compiler .......................... Clang 18.1.3, /usr/bin/clang++
Preprocessor defined macros ........... ABORT_ON_FAILED_ASSUME
Linker flags .......................... -fsanitize=memory -fsanitize-memory-track-origins=2 -fno-omit-frame-pointer -g -O1 -fno-optimize-sibling-calls -nostdinc++ -nostdlib++ -isystem /msan/cxx_build/include/c++/v1 -L/msan/cxx_build/lib -Wl,-rpath,/msan/cxx_build/lib -lc++ -lc++abi -lpthread -Wno-unused-command-line-argument -O2 -g -fsanitize=fuzzer,memory -fstack-protector-all -fcf-protection=full -fstack-clash-protection -Wl,-z,relro -Wl,-z,now -Wl,-z,separate-code -fPIE -pie

Not sure why that would happen, but I'm assuming that is related.

https://cirrus-ci.com/task/5304183451025408:

Cross compiling ....................... FALSE
C++ compiler .......................... Clang 18.1.3, /usr/bin/clang++
CMAKE_BUILD_TYPE ...................... RelWithDebInfo
Preprocessor defined macros ........... ABORT_ON_FAILED_ASSUME
C++ compiler flags .................... -fsanitize=memory -fsanitize-memory-track-origins=2 -fno-omit-frame-pointer -g -O1 -fno-optimize-sibling-calls -nostdinc++ -nostdlib++ -isystem /msan/cxx_build/include/c++/v1 -L/msan/cxx_build/lib -Wl,-rpath,/msan/cxx_build/lib -lc++ -lc++abi -lpthread -Wno-unused-command-line-argument -O2 -g -std=c++20 -fPIC -fdebug-prefix-map=/ci_container_base=. -fmacro-prefix-map=/ci_container_base=. -Werror -fsanitize=fuzzer,memory -Wall -Wextra -Wgnu -Wformat -Wformat-security -Wvla -Wshadow-field -Wthread-safety -Wloop-analysis -Wredundant-decls -Wunused-member-function -Wdate-time -Wconditional-uninitialized -Woverloaded-virtual -Wsuggest-override -Wimplicit-fallthrough -Wunreachable-code -Wdocumentation -Wself-assign -Wundef -Wno-unused-parameter -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -Wstack-protector -fstack-protector-all -fcf-protection=full -fstack-clash-protection -DBOOST_MULTI_INDEX_ENABLE_SAFE_MODE -U_FORTIFY_SOURCE
Linker flags .......................... -fsanitize=memory -fsanitize-memory-track-origins=2 -fno-omit-frame-pointer -g -O1 -fno-optimize-sibling-calls -nostdinc++ -nostdlib++ -isystem /msan/cxx_build/include/c++/v1 -L/msan/cxx_build/lib -Wl,-rpath,/msan/cxx_build/lib -lc++ -lc++abi -lpthread -Wno-unused-command-line-argument -O2 -g -fsanitize=fuzzer,memory -fstack-protector-all -fcf-protection=full -fstack-clash-protection -Wl,-z,relro -Wl,-z,now -Wl,-z,separate-code -fPIE -pie