Skeleton swpp-compiler executable binary segfaults on macOS

[cr0sh]

Tested on: https://github.com/snu-sf-class/swpp202401-compiler/commit/30aaa689cf7b1beab60fab9655bd25c1db6111ac, https://github.com/snu-sf-class/swpp202401-compiler/commit/d3e9960260d9c4618bab8228fa29ed6cd3678bf9

Environment: LLVM 18.1.0 (https://github.com/llvm/llvm-project/commit/461274b81d8641eab64d494accddc81d7db8a09e), macOS 14.4.1, Apple M1 Pro (AArch64)

Steps to reproduce:

• Clone template compiler repository with given commit hashes
• ./compile.sh
• build/swpp-compiler test.ll test.s and it immediately segfaults(presumably due to a null ptr deref).

Sample backtrace from LLDB:

  Process 50761 stopped
  * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x20)
      frame #0: 0x00000001034a3008 libSComp.dylib`llvm::AnalysisManager<llvm::Module>::getResultImpl(llvm::AnalysisKey*, llvm::Module&) + 576
  libSComp.dylib`llvm::AnalysisManager<llvm::Module>::getResultImpl:
  ->  0x1034a3008 <+576>: ldr    x9, [x8, #0x10]
      0x1034a300c <+580>: add    x8, sp, #0x10
      0x1034a3010 <+584>: mov    x0, x23
      0x1034a3014 <+588>: mov    x1, x21
  (lldb) bt
  * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x20)
    * frame #0: 0x00000001034a3008 libSComp.dylib`llvm::AnalysisManager<llvm::Module>::getResultImpl(llvm::AnalysisKey*, llvm::Module&) + 576
      frame #1: 0x00000001011a8414 libSCOpt.dylib`llvm::ModuleToPostOrderCGSCCPassAdaptor::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) + 572
      frame #2: 0x00000001034a0b14 libSComp.dylib`llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) + 368
      frame #3: 0x00000001011a2e1c libSCOpt.dylib`sc::opt::optimizeIR(std::__1::unique_ptr<llvm::Module, std::__1::default_delete<llvm::Module>>&&, llvm::AnalysisManager<llvm::Module>&) + 772
      frame #4: 0x000000010229bf44 libSComp.dylib`sc::compile(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::basic_string_view<char, std::__1::char_traits<char>>, bool) + 1172
      frame #5: 0x0000000100001900 swpp-compiler`main + 392
      frame #6: 0x00000001855ba0e0 dyld`start + 2360

Sample test.ll file:

define i64 @add_0() {
; CHECK-LABEL: @add_0(
; CHECK-NEXT:  %1 = mul i64 1, 1
  %1 = mul i64 1, 1
; CHECK-NEXT: ret i64 %1
  %2 = add i64 %1, 0
  ret i64 %2
}

This bug is affecting the use of any predefined analysis from LLVM as the AnalysisManager::getResultImpl is the main segfault point.

[strikef]

LLVM is not API-stable. That is, we cannot guarantee that our codebase works properly on other versions of LLVM. Please build & run the compiler based on the designated LLVM version (18.1.0), and advise TAs if the problem persists.

[cr0sh]

Confirmed with LLVM 18.1.0, with -fsanitize=address,undefined and an UBSan alert was triggered(it also segfaults without sanitizers)

/Users/namjh/dev/personal/swpp2024/llvm-project/build/bin/../include/c++/v1/variant:495:12: runtime error: call to function decltype(auto) std::__1::__variant_detail::__visitation::__base::__dispatcher<0ul, 0ul>::__dispatch[abi:ne180100]<void std::__1::__variant_detail::__ctor<std::__1::__variant_detail::__traits<sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>>::__generic_construct[abi:ne180100]<std::__1::__variant_detail::__move_constructor<std::__1::__variant_detail::__traits<sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>, (std::__1::__variant_detail::_Trait)1>>(std::__1::__variant_detail::__ctor<std::__1::__variant_detail::__traits<sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>>&, std::__1::__variant_detail::__move_constructor<std::__1::__variant_detail::__traits<sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>, (std::__1::__variant_detail::_Trait)1>&&)::'lambda'(std::__1::__variant_detail::__move_constructor<std::__1::__variant_detail::__traits<sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>, (std::__1::__variant_detail::_Trait)1>&, auto&&)&&, std::__1::__variant_detail::__base<(std::__1::__variant_detail::_Trait)1, sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>&, std::__1::__variant_detail::__base<(std::__1::__variant_detail::_Trait)1, sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>&&>(std::__1::__variant_detail::__move_constructor<std::__1::__variant_detail::__traits<sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>, (std::__1::__variant_detail::_Trait)1>, std::__1::__variant_detail::__base<(std::__1::__variant_detail::_Trait)1, sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>&, std::__1::__variant_detail::__base<(std::__1::__variant_detail::_Trait)1, sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>&&) through pointer to incorrect function type 'void (*)((lambda at /Users/namjh/dev/personal/swpp2024/llvm-project/build/bin/../include/c++/v1/variant:814:11) &&, std::__variant_detail::__base<std::__variant_detail::_Trait::_Available, sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel> &, std::__variant_detail::__base<std::__variant_detail::_Trait::_Available, sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel> &&)'
(swpp-compiler:arm64+0x1003ef130): note: decltype(auto) std::__1::__variant_detail::__visitation::__base::__dispatcher<0ul, 0ul>::__dispatch[abi:ne180100]<void std::__1::__variant_detail::__ctor<std::__1::__variant_detail::__traits<sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>>::__generic_construct[abi:ne180100]<std::__1::__variant_detail::__move_constructor<std::__1::__variant_detail::__traits<sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>, (std::__1::__variant_detail::_Trait)1>>(std::__1::__variant_detail::__ctor<std::__1::__variant_detail::__traits<sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>>&, std::__1::__variant_detail::__move_constructor<std::__1::__variant_detail::__traits<sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>, (std::__1::__variant_detail::_Trait)1>&&)::'lambda'(std::__1::__variant_detail::__move_constructor<std::__1::__variant_detail::__traits<sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>, (std::__1::__variant_detail::_Trait)1>&, auto&&)&&, std::__1::__variant_detail::__base<(std::__1::__variant_detail::_Trait)1, sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>&, std::__1::__variant_detail::__base<(std::__1::__variant_detail::_Trait)1, sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>&&>(std::__1::__variant_detail::__move_constructor<std::__1::__variant_detail::__traits<sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>, (std::__1::__variant_detail::_Trait)1>, std::__1::__variant_detail::__base<(std::__1::__variant_detail::_Trait)1, sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>&, std::__1::__variant_detail::__base<(std::__1::__variant_detail::_Trait)1, sc::backend::symbol::base::Register, sc::backend::symbol::base::Argument, sc::backend::symbol::base::StackPtr, sc::backend::symbol::base::Constant, sc::backend::symbol::base::VectorRegister, sc::backend::symbol::base::FunctionName, sc::backend::symbol::base::BasicBlockLabel>&&) defined here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /Users/namjh/dev/personal/swpp2024/llvm-project/build/bin/../include/c++/v1/variant:495:12 

[cr0sh]

Backtrace from LLDB:

(lldb) target create "build/swpp-compiler"
Current executable set to '/Users/namjh/dev/personal/swpp2024/swpp202401-compiler-team4/build/swpp-compiler' (arm64).
(lldb) settings set -- target.run-args  "test.ll" "test.s"
(lldb) r
Process 94801 launched: '/Users/namjh/dev/personal/swpp2024/swpp202401-compiler-team4/build/swpp-compiler' (arm64)
swpp-compiler(94801,0x1ed64fac0) malloc: nano zone abandoned due to inability to reserve vm space.
2024-05-20 15:48:48.303523+0900 swpp-compiler[94801:7269640] dynamic_cast error 2: One or more of the following type_info's has hidden visibility or is defined in more than one translation unit. They should all have public visibility. St9type_info, N10__cxxabiv120__si_class_type_infoE, N10__cxxabiv117__class_type_infoE.
2024-05-20 15:48:48.303883+0900 swpp-compiler[94801:7269640] dynamic_cast error 2: One or more of the following type_info's has hidden visibility or is defined in more than one translation unit. They should all have public visibility. St9type_info, N10__cxxabiv121__vmi_class_type_infoE, N10__cxxabiv117__class_type_infoE.
2024-05-20 15:48:48.303897+0900 swpp-compiler[94801:7269640] dynamic_cast error 2: One or more of the following type_info's has hidden visibility or is defined in more than one translation unit. They should all have public visibility. St9type_info, N10__cxxabiv121__vmi_class_type_infoE, N10__cxxabiv117__class_type_infoE.
2024-05-20 15:48:48.303910+0900 swpp-compiler[94801:7269640] dynamic_cast error 2: One or more of the following type_info's has hidden visibility or is defined in more than one translation unit. They should all have public visibility. N10__cxxabiv117__class_type_infoE, N10__cxxabiv121__vmi_class_type_infoE, N10__cxxabiv121__vmi_class_type_infoE.
2024-05-20 15:48:48.303933+0900 swpp-compiler[94801:7269640] dynamic_cast error 2: One or more of the following type_info's has hidden visibility or is defined in more than one translation unit. They should all have public visibility. St9type_info, N10__cxxabiv120__si_class_type_infoE, N10__cxxabiv117__class_type_infoE.
Process 94801 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x00000001034ce8dc libSComp.dylib`llvm::AnalysisManager<llvm::Module>::getResultImpl(llvm::AnalysisKey*, llvm::Module&) + 572
libSComp.dylib`llvm::AnalysisManager<llvm::Module>::getResultImpl:
->  0x1034ce8dc <+572>: ldr    x8, [x23]
    0x1034ce8e0 <+576>: ldr    x9, [x8, #0x10]
    0x1034ce8e4 <+580>: add    x8, sp, #0x10
    0x1034ce8e8 <+584>: mov    x0, x23
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x00000001034ce8dc libSComp.dylib`llvm::AnalysisManager<llvm::Module>::getResultImpl(llvm::AnalysisKey*, llvm::Module&) + 572
    frame #1: 0x0000000101a08968 libSCOpt.dylib`llvm::ModuleToPostOrderCGSCCPassAdaptor::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) + 572
    frame #2: 0x00000001034cc3ec libSComp.dylib`llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module>>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) + 368
    frame #3: 0x00000001019f7abc libSCOpt.dylib`sc::opt::optimizeIR(std::__1::unique_ptr<llvm::Module, std::__1::default_delete<llvm::Module>>&&, llvm::AnalysisManager<llvm::Module>&) + 1504
    frame #4: 0x00000001022bce7c libSComp.dylib`sc::compile(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::basic_string_view<char, std::__1::char_traits<char>>, bool) + 3564
    frame #5: 0x00000001000021f8 swpp-compiler`main + 864
    frame #6: 0x00000001855ba0e0 dyld`start + 2360

[cr0sh]

Also trying in the GHA environment.

[strikef]

As of now it seems that this issue is Mac-specific. By the way, does the skeleton compile at all? We just noticed that the compiler skeleton fails to link as one of the LLVM library was not included in the build script.

But other than that, we are still unable to reproduce the issue.

[cr0sh]

If you are trying to build with debug mode enabled, libc++ linkage is missing so I fixed in downstream. (Please see https://github.com/foxisobese/swpp202401-compiler-team4/pull/14). Would happy to help if there's another issue.

edit: typo

[cr0sh]

As of now it seems that this issue is Mac-specific.

Confirmed by checking on the provided GHA env which run well(https://github.com/foxisobese/swpp202401-compiler-team4/actions/runs/9155302229/job/25167398500)

[strikef]

Out of curiosity: When you compiled LLVM using the script we provided, did <llvm-install-dir>/lib not include libc++.dylib? When I installed LLVM using the script, it already had libc++.dylib inside that directory, and c++ directory did not exist at all (although in Linux libc++.so resided inside c++ directory).

[cr0sh]

To be honest I reused an existing homebrew installation.

[strikef]

When you look at the provided LLVM build script, we configured a bootstrap build to ensure that the built LLVM also depends on the same c++ library as the other programs link to. This is because the compiler codebase relies on the LLVM, and if LLVM and the codebase use different c++ library, the code will fail to link.

Also, frankly speaking, we haven't tested the compiler on LLVM other than the one built with our script. Please compile the LLVM using the class repo script and try building the compiler again.

[cr0sh]

Okay, I'm rebuilding the entire LLVM toolchain with params extracted from install-llvm.sh(but skipping bootstrapping due to lack of time and space resources, with the homebrew LLVM). Will report the result here.

[cr0sh]

I managed to build LLVM with the script given, but the linking procedure fails to locate llvm::ReplaceInstWithInst(llvm::Instruction*, llvm::Instruction*), which is defined in libLLVMTransfromUtils.dylib. After adding transformutils to llvm_map_components_to_libnames(backend_llvm_libs analysis scalaropts), compilation succeeds and no runtime crashes are happened. Thank you for your help!

[strikef]

I managed to build LLVM with the script given, but the linking procedure fails to locate llvm::ReplaceInstWithInst(llvm::Instruction*, llvm::Instruction*), which is defined in libLLVMTransfromUtils.dylib. After adding transformutils to llvm_map_components_to_libnames(backend_llvm_libs analysis scalaropts), compilation succeeds and no runtime crashes are happened. Thank you for your help!

I also noticed this issue while reproducing your problem. This issue has been addressed in the latest patch (2024.1.9)

[mscheong01]

@strikef The error seems to persist after applying 2024.1.9 patch. 😢

ld64.lld: error: undefined symbol: llvm::ReplaceInstWithInst(llvm::Instruction*, llvm::Instruction*)