gollvm/passes: llvm-goc crashes when enable-gc is set

[hanchaoqun]

What version of Go are you using (go version)?

gollvm master

$ go version
go version unknown linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env

What did you do?

File a.go

package aaa

var IsAlphaCase = false

func IsStubForAlpha() bool {
    return IsAlphaCase
}

compile a.go with llvm-goc -enable-gc=1 a.go command.

What did you expect to see?

No error, successful build.

What did you see instead?

han@DESKTOP-2MU542E:/mnt/d/gobuild/test$ llvm-goc -enable-gc=1 a.go
 #0 0x000055ad8cab1678 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /mnt/d/gobuild/gollvm/llvm-project/llvm/lib/Support/Unix/Signals.inc:565:0
 #1 0x000055ad8cab172f PrintStackTraceSignalHandler(void*) /mnt/d/gobuild/gollvm/llvm-project/llvm/lib/Support/Unix/Signals.inc:632:0
 #2 0x000055ad8caaf3e3 llvm::sys::RunSignalHandlers() /mnt/d/gobuild/gollvm/llvm-project/llvm/lib/Support/Signals.cpp:97:0
 #3 0x000055ad8cab0ff9 SignalHandler(int) /mnt/d/gobuild/gollvm/llvm-project/llvm/lib/Support/Unix/Signals.inc:407:0
 #4 0x00007f228f016980 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x12980)
 #5 0x000055ad8d4b0987 llvm::EHStreamer::emitExceptionTable() /mnt/d/gobuild/gollvm/llvm-project/llvm/lib/CodeGen/AsmPrinter/EHStreamer.cpp:735:0
 #6 0x000055ad8d439faf llvm::DwarfCFIException::endFunction(llvm::MachineFunction const*) /mnt/d/gobuild/gollvm/llvm-project/llvm/lib/CodeGen/AsmPrinter/DwarfCFIException.cpp:181:0
 #7 0x000055ad8d40a638 llvm::AsmPrinter::emitFunctionBody() /mnt/d/gobuild/gollvm/llvm-project/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp:1504:0
 #8 0x000055ad8babcb63 llvm::X86AsmPrinter::runOnMachineFunction(llvm::MachineFunction&) /mnt/d/gobuild/gollvm/llvm-project/llvm/lib/Target/X86/X86AsmPrinter.cpp:85:0
 #9 0x000055ad8c07903f llvm::MachineFunctionPass::runOnFunction(llvm::Function&) /mnt/d/gobuild/gollvm/llvm-project/llvm/lib/CodeGen/MachineFunctionPass.cpp:72:0
#10 0x000055ad8c769668 llvm::FPPassManager::runOnFunction(llvm::Function&) /mnt/d/gobuild/gollvm/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1439:0
#11 0x000055ad8c769911 llvm::FPPassManager::runOnModule(llvm::Module&) /mnt/d/gobuild/gollvm/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1485:0
#12 0x000055ad8c769d39 (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) /mnt/d/gobuild/gollvm/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1554:0
#13 0x000055ad8c764db9 llvm::legacy::PassManagerImpl::run(llvm::Module&) /mnt/d/gobuild/gollvm/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:542:0
#14 0x000055ad8c76a5c3 llvm::legacy::PassManager::run(llvm::Module&) /mnt/d/gobuild/gollvm/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1682:0
#15 0x000055ad8a187a31 gollvm::driver::CompileGoImpl::invokeBackEnd(gollvm::driver::Action const&) /mnt/d/gobuild/gollvm/llvm-project/llvm/tools/gollvm/driver/CompileGo.cpp:980:0
#16 0x000055ad8a1840c5 gollvm::driver::CompileGoImpl::performAction(gollvm::driver::Compilation&, gollvm::driver::Action const&, llvm::SmallVector<gollvm::driver::Artifact*, 3u> const&, gollvm::driver::Artifact const&) /mnt/d/gobuild/gollvm/llvm-project/llvm/tools/gollvm/driver/CompileGo.cpp:195:0
#17 0x000055ad8a187bda gollvm::driver::CompileGo::performAction(gollvm::driver::Compilation&, gollvm::driver::Action const&, llvm::SmallVector<gollvm::driver::Artifact*, 3u> const&, gollvm::driver::Artifact const&) /mnt/d/gobuild/gollvm/llvm-project/llvm/tools/gollvm/driver/CompileGo.cpp:1004:0
#18 0x000055ad8a177382 gollvm::driver::Driver::processAction(gollvm::driver::Action*, gollvm::driver::Compilation&, bool) /mnt/d/gobuild/gollvm/llvm-project/llvm/tools/gollvm/driver/Driver.cpp:706:0
#19 0x000055ad8a177545 gollvm::driver::Driver::processActions(gollvm::driver::Compilation&) /mnt/d/gobuild/gollvm/llvm-project/llvm/tools/gollvm/driver/Driver.cpp:729:0
#20 0x000055ad8a16ac7c main /mnt/d/gobuild/gollvm/llvm-project/llvm/tools/gollvm/driver-main/llvm-goc.cpp:228:0
#21 0x00007f228dcaabf7 __libc_start_main /build/glibc-S9d2JN/glibc-2.27/csu/../csu/libc-start.c:344:0
#22 0x000055ad8a16a06a _start (/mnt/d/gobuild/gollvm/bin.debug/bin/llvm-goc+0x91a06a)
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
Stack dump:
0.      Program arguments: /mnt/d/gobuild/gollvm/bin.debug/bin/llvm-goc -enable-gc=1 a.go
1.      Running pass 'Function Pass Manager' on module 'gomodule'.
2.      Running pass 'X86 Assembly Printer' on function '@go_0aaa.IsStubForAlpha'
Segmentation fault

[hanchaoqun]

I'll submit a fix for this issue soon.

cc @cherrymui

[huanglanzhiguan]

gollvm doesn't work when enable_gc is enabled, since there is no stackmap generated for the runtime functions written in C/C++, and gollvm uses sysv64ABI to generate code, the CSRs are not handled during the unwinding process. I think there's still lots of work to do.

[hanchaoqun]

gollvm doesn't work when enable_gc is enabled, since there is no stackmap generated for the runtime functions written in C/C++, and gollvm uses sysv64ABI to generate code, the CSRs are not handled during the unwinding process. I think there's still lots of work to do.

By looking into the code, i think it already has emits the pointer map for Go func, and when GC is triggered by runtime, it can get that infomation from the unwinding process. Yes, you are right, no for C funcs , so C funcs are skipped when unwinding. And I don't see any plans to support GC for C functions.

[ianlancetaylor]

There is no need for GoLLVM to build stackmaps for C functions. The rules for Go calling into C do not permit Go pointers to live only in C code.

[huanglanzhiguan]

@ianlancetaylor Thanks, how about the pointers lived in the callee saved registers? BTW what's the current status of accurate GC in GoLLVM?

[ianlancetaylor]

My understanding is that GoLLVM has fully accurate GC. I don't know the details of how callee saved registers are handled. @cherrymui did the implementation.

[huanglanzhiguan]

@cherrymui Hi, could you please spare us some time talking about the status of the accurate GC implementation of GoLLVM. Is it already worked and well tested? We've tried to enable it and run some benchmarks but failed. By looking into the code, I didn't see any code related to handling pointers in the callee saved registers, and the LLVM MC didn't emit the information of this part. The current implementation only emits the stackmap entry, which is a bit vector of live pointers on the stack:

      // Stack map entry:
      //   uint32_t nbits;
      //   uint8_t *data;

Similar to the go compiler's implementation, but I think there's much differences between these two compilers. The go compiler doesn't have any callee saved registers, and pointers live in the registers are well handled by asyncpreempt(save all registers on stack and scan them conservatively). I don't have much knowledge & background about the GC implementation, please correct me if I missed something. Thanks! cc @ianlancetaylor

[hanchaoqun]

@huanglanzhiguan gollvm implement the statepoint pass, which will analyze the live pointers, and then spill the live pointers which across the statepoint into current stack slots.

[hanchaoqun]

The fix CL : https://go-review.googlesource.com/c/gollvm/+/407274

[cherrymui]

The GC implementation worked and was well tested. But I don't test it on a frequent base.

The idea is that for any pointers that are live across a call, the compiler would generate instructions to spill the pointers (or perhaps pointers to the same object) to the stack before the call, so there is always a stack copy which the GC can see. And then the GC doesn't need to look at callee-save registers. It is probably less efficient than scanning callee-save registers, but it was easier to implement.

[MATRIXKOO]

@cherrymui By the way, if I may ask, what is the reason behind gollvm not utilizing LLVM's stack map?

[cherrymui]

@MATRIXKOO I'm not sure I understand what you mean by "gollvm not utilizing LLVM's stack map". Gollvm does use LLVM's stack maps feature when -enable-gc is set.

[MATRIXKOO]

@cherrymui I mean llvm generated stackmap, located in llvm-project/llvm/tools/gollvm/passes/GC.cpp，Someone told me that gollvm don't use llvm stackmap. He/She said, "gollvm don't support stackmap." true?

[cherrymui]

llvm-project/llvm/tools/gollvm/passes/GC.cpp

This is exactly what gollvm uses.

It is true that gollvm doesn't use the stack map format mentioned in https://llvm.org/docs/StackMaps.html . That format contains much more information than what we need. We use a more compact format.