Symbolizer crashes when compiling binutils

[chenju2k6]

Binutils version - 2.33.1 LLVM version - 10 OS: Ubuntu 18.04

The below assertion fails during compilation

assert(!verifyFunction(F, &errs()))

[sebastianpoeplau]

Hi @chenju2k6,

That's interesting and most likely a bug in SymCC. (The assertion checks whether we produce valid bitcode.) Could you share the logs just before the error? In particular, which file triggers the error, and what is the compiler invocation? That would make it easier for me to debug the issue...

[chenju2k6]

Hi Sebastian,

Thanks for the reply. Below is the log. Besides, I have a pull request which seems to fix the issue, but I am not sure it is a correct fix. The error is triggered in the compilation process for binutils-2.33.1

Symbolizing function md_assemble
Warning: unhandled LLVM intrinsic llvm.dbg.label
Warning: unhandled LLVM intrinsic llvm.dbg.label
Warning: unhandled LLVM intrinsic llvm.dbg.label
Warning: unhandled LLVM intrinsic llvm.dbg.label
Warning: unhandled LLVM intrinsic llvm.dbg.label
Warning: unhandled LLVM intrinsic llvm.dbg.label
Warning: unhandled LLVM intrinsic llvm.dbg.label
Warning: unhandled LLVM intrinsic llvm.dbg.label
Warning: unhandled LLVM intrinsic llvm.dbg.label
Warning: unhandled LLVM intrinsic llvm.dbg.label
Warning: unhandled LLVM intrinsic llvm.dbg.label
Warning: unhandled LLVM intrinsic llvm.dbg.label
Warning: unhandled LLVM intrinsic llvm.dbg.label
Warning: unhandled LLVM intrinsic llvm.dbg.label
Warning: losing track of symbolic expressions at bit-count operation   %13744 = call i32 @llvm.ctpop.i32(i32 %13743) #15, !dbg !11972, !range !11973
Warning: unhandled LLVM intrinsic llvm.dbg.label
Type too small for ZExt
  %14247 = zext i80 %13978 to i64, !dbg !10112
Type too small for ZExt
  %14252 = zext i80 %14204 to i64, !dbg !10112
Type too small for ZExt
  %14285 = zext i80 %13915 to i64, !dbg !10113
Type too small for ZExt
  %30023 = zext i80 %30010 to i64, !dbg !11830
Type too small for ZExt
  %30028 = zext i80 %30017 to i64, !dbg !11830
Type too small for ZExt
  %33961 = zext i80 %33957 to i64, !dbg !12419
Type too small for ZExt
  %33969 = zext i80 %33931 to i64, !dbg !12421
clang: ../compiler/Pass.cpp:86: virtual bool SymbolizePass::runOnFunction(llvm::Function&): Assertion `!verifyFunction(F, &errs()) && "SymbolizePass produced invalid bitcode"' failed.
Stack dump:
0.  Program arguments: /usr/local/llvm-10/bin/clang -Xclang -load -Xclang /home/cju/symcc/build/libSymbolize.so -DHAVE_CONFIG_H -I. -I. -I. -I../bfd -I./config -I./../include -I./.. -I./../bfd -DLOCALEDIR="/usr/local/share/locale" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Wwrite-strings -I./../zlib -g -O2 -MT config/tc-i386.o -MD -MP -MF config/.deps/tc-i386.Tpo -c -o config/tc-i386.o config/tc-i386.c -L/home/cju/symcc/build/SymRuntime-prefix/src/SymRuntime-build -lSymRuntime -Wl,-rpath,/home/cju/symcc/build/SymRuntime-prefix/src/SymRuntime-build -Qunused-arguments
1.  <eof> parser at end of file
2.  Per-module optimization passes
3.  Running pass 'Function Pass Manager' on module 'config/tc-i386.c'.
4.  Running pass 'Symbolization Pass' on function '@md_assemble'
 #0 0x00007f8769d2ce3a llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/usr/local/llvm-10/bin/../lib/libLLVMSupport.so.10+0x1b3e3a)
 #1 0x00007f8769d2a6c4 llvm::sys::RunSignalHandlers() (/usr/local/llvm-10/bin/../lib/libLLVMSupport.so.10+0x1b16c4)
 #2 0x00007f8769d2a935 llvm::sys::CleanupOnSignal(unsigned long) (/usr/local/llvm-10/bin/../lib/libLLVMSupport.so.10+0x1b1935)
 #3 0x00007f8769c3a100 CrashRecoverySignalHandler(int) (/usr/local/llvm-10/bin/../lib/libLLVMSupport.so.10+0xc1100)
 #4 0x00007f876e0578a0 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x128a0)
 #5 0x00007f8768b30f47 raise /build/glibc-2ORdQG/glibc-2.27/signal/../sysdeps/unix/sysv/linux/raise.c:51:0
 #6 0x00007f8768b328b1 abort /build/glibc-2ORdQG/glibc-2.27/stdlib/abort.c:81:0
 #7 0x00007f8768b2242a __assert_fail_base /build/glibc-2ORdQG/glibc-2.27/assert/assert.c:89:0
 #8 0x00007f8768b224a2 (/lib/x86_64-linux-gnu/libc.so.6+0x304a2)
 #9 0x00007f875d26bd61 SymbolizePass::runOnFunction(llvm::Function&) (/home/cju/symcc/build/libSymbolize.so+0x6ed61)
#10 0x00007f876b2f7bc2 llvm::FPPassManager::runOnFunction(llvm::Function&) (/usr/local/llvm-10/bin/../lib/libLLVMCore.so.10+0x1bebc2)
#11 0x00007f876b2f85c1 llvm::FPPassManager::runOnModule(llvm::Module&) (/usr/local/llvm-10/bin/../lib/libLLVMCore.so.10+0x1bf5c1)
#12 0x00007f876b2f8a3c llvm::legacy::PassManagerImpl::run(llvm::Module&) (/usr/local/llvm-10/bin/../lib/libLLVMCore.so.10+0x1bfa3c)
#13 0x00007f876d9901dd (anonymous namespace)::EmitAssemblyHelper::EmitAssembly(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) (/usr/local/llvm-10/bin/../lib/libclangCodeGen.so.10+0xb81dd)
#14 0x00007f876d991eab clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) (/usr/local/llvm-10/bin/../lib/libclangCodeGen.so.10+0xb9eab)
#15 0x00007f876dc83374 clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext&) (/usr/local/llvm-10/bin/../lib/libclangCodeGen.so.10+0x3ab374)
#16 0x00007f87639a5b99 clang::ParseAST(clang::Sema&, bool, bool) (/usr/local/llvm-10/bin/../lib/../lib/libclangParse.so.10+0x33b99)

[chenju2k6]

Looks like there's a crash running binutil program nm. Below is the stack (partial)

Thread 1 "nm_symcc" received signal SIGSEGV, Segmentation fault.
0x00007ffff79129dc in qsym::equalMetadata(qsym::Expr const&, qsym::Expr const&) () from /home/cju/symcc/build/SymRuntime-prefix/src/SymRuntime-build/libSymRuntime.so
(gdb) bt
#0  0x00007ffff79129dc in qsym::equalMetadata(qsym::Expr const&, qsym::Expr const&) () from /home/cju/symcc/build/SymRuntime-prefix/src/SymRuntime-build/libSymRuntime.so
#1  0x00007ffff7912a2a in qsym::equalShallowly(qsym::Expr const&, qsym::Expr const&) () from /home/cju/symcc/build/SymRuntime-prefix/src/SymRuntime-build/libSymRuntime.so
#2  0x00007ffff7938d33 in qsym::WeakExprRefEqual::operator()(std::weak_ptr<qsym::Expr>, std::weak_ptr<qsym::Expr>) const () from /home/cju/symcc/build/SymRuntime-prefix/src/SymRuntime-build/libSymRuntime.so

[sebastianpoeplau]

Does the crash occur with the fix you suggested in #13? I suppose nm wouldn't compile without it, right?

[chenju2k6]

Sorry for the late reply. The crash occurs with fix #13. I mean all Binutils programs (including nm) can compile with fix #13 but all of them will crash during run time.

[sebastianpoeplau]

Great, thanks for the details! I can reproduce the problem locally - the problem is indeed the 80-bit integer, as discussed in #13. It turns out that SymCC doesn't expect to see that bit width in integer literals. However, since we already have handling for 128-bit literals, I think I can just reuse the logic for anything between 65 and 128 bits. Will push the fix shortly...

[sebastianpoeplau]

@chenju2k6 all binutils compile cleanly for me now, and running nm produces the expected results (including new inputs from the solver). Please feel free to reopen the issue if something doesn't work on your side.

[chenju2k6]

@sebastianpoeplau Thank you! I just tried the fix, Binutils compile cleanly. But seems objdump/nm/size crashes when analyzing small_exec.elf (a seed found from AFL). I built with -DQSYM_BACKEND=ON The Binutils version is 2.33.1

The command to produce the issue is below (for objdump)

SYMCC_INPUT_FILE=/path/to/small_exec.elf ./objdump -D /path/to/small_exec.elf

[sebastianpoeplau]

@chenju2k6 yep, I see the same problem - looking into it. Since it seems unrelated to the compilation error, I'll open a new issue.