search

golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
123.99k stars 17.67k forks source link

runtime: Crash in GC on Debian riscv64 (qemu) #54104

Closed phpeterson-usf closed 2 years ago

phpeterson-usf commented 2 years ago

What version of Go are you using (go version)?

$ go version
go version go1.18.4 darwin/arm64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

I am cross-compiling the micro editor on my Mac targeting a Debian riscv64 guest under QEMU. The Debian guest is Linux debian 5.18.0-2-riscv64 https://github.com/zyedidia/micro/issues/1 SMP Debian 5.18.5-1 (2022-06-16) riscv64 GNU/Linux

go env Output
$ go env
GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/phil/Library/Caches/go-build"
GOENV="/Users/phil/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/phil/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/phil/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.18.4"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/phil/riscv64-ubuntu/micro/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/bl/ncrkxg1d6r130_5pb8mn3y3m0000gn/T/go-build3611809763=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

  1. After cloning the repo, the build command is GOOS=linux GOARCH=riscv64 CGO_ENABLED=0 make build.
  2. The resulting binary is called micro and I scp that to the Debian guest

What did you expect to see?

I expect to run the micro editor

What did you see instead?

The micro editor crashes on launch with the output below, apparently showing a crash in the Go GC. I opened an issue in the micro repo, but the author suggests that the root cause is inside Go.

Interestingly, the same micro binary runs on an Ubuntu guest Linux ubuntu 5.13.0-1026-generic #29~20.04.1-Ubuntu SMP Fri Jun 3 11:55:52 UTC 2022 riscv64 riscv64 riscv64 GNU/Linux but crashes on launch on the Debian guest.

stack trace stack trace

runtime: lfstack.push invalid packing: node=0xffffff8cc2b980 cnt=0x1 packed=0xffff8cc2b9800001 -> node=0xffff8cc2b980
fatal error: lfstack.push

runtime stack:
runtime.throw({0x5966ba, 0xc})
    runtime/panic.go:992 +0x58
runtime.(*lfstack).push(0xaf62e8, 0xffffff8cc2b980)
    runtime/lfstack.go:30 +0x144
runtime.(*spanSetBlockAlloc).free(...)
    runtime/mspanset.go:292
runtime.(*spanSet).reset(0xaef898)
    runtime/mspanset.go:265 +0xac
runtime.finishsweep_m()
    runtime/mgcsweep.go:260 +0xc8
runtime.gcStart.func1()
    runtime/mgc.go:664 +0x1c
runtime.systemstack()
    runtime/asm_riscv64.s:133 +0x50

goroutine 1 [running]:
runtime.systemstack_switch()
    runtime/asm_riscv64.s:96 +0x8 fp=0xc0004b3720 sp=0xc0004b3718 pc=0x74098
runtime.gcStart({0x0, 0x0, 0x0})
    runtime/mgc.go:663 +0x500 fp=0xc0004b3798 sp=0xc0004b3720 pc=0x29188
runtime.mallocgc(0x30, 0x5576e0, 0x1)
    runtime/malloc.go:1205 +0x71c fp=0xc0004b3810 sp=0xc0004b3798 pc=0x1bdb4
runtime.growslice(0x5576e0, {0x0, 0x0, 0x0}, 0x1)
    runtime/slice.go:278 +0x4e0 fp=0xc0004b3860 sp=0xc0004b3810 pc=0x5e4b8
regexp/syntax.(*compiler).inst(...)
    regexp/syntax/compile.go:164
regexp/syntax.(*compiler).init(...)
    regexp/syntax/compile.go:83
regexp/syntax.Compile(0xc000131810)
    regexp/syntax/compile.go:73 +0x9c fp=0xc0004b3920 sp=0xc0004b3860 pc=0x151214
regexp.compile({0x59c8fd, 0x14}, 0xd4, 0x0)
    regexp/regexp.go:180 +0xa4 fp=0xc0004b39b8 sp=0xc0004b3920 pc=0x167c94
regexp.Compile(...)
    regexp/regexp.go:135
regexp.MustCompile({0x59c8fd, 0x14})
    regexp/regexp.go:315 +0x3c fp=0xc0004b3a38 sp=0xc0004b39b8 pc=0x1688ac
main.LoadInput({0xc00001c310, 0x0, 0x0})
    github.com/zyedidia/micro/v2/cmd/micro/micro.go:166 +0x14c fp=0xc0004b3bb8 sp=0xc0004b3a38 pc=0x4af5d4
main.main()
    github.com/zyedidia/micro/v2/cmd/micro/micro.go:319 +0x748 fp=0xc0004b3f80 sp=0xc0004b3bb8 pc=0x4b0988
runtime.main()
    runtime/proc.go:250 +0x228 fp=0xc0004b3fd8 sp=0xc0004b3f80 pc=0x48480
runtime.goexit()
    runtime/asm_riscv64.s:497 +0x4 fp=0xc0004b3fd8 sp=0xc0004b3fd8 pc=0x75ec4

goroutine 6 [sleep]:
time.Sleep(0x1dcd65000)
    runtime/time.go:194 +0x150
github.com/zyedidia/micro/v2/internal/buffer.backupThread()
    github.com/zyedidia/micro/v2/internal/buffer/backup.go:37 +0x28
created by github.com/zyedidia/micro/v2/internal/buffer.init.0
    github.com/zyedidia/micro/v2/internal/buffer/backup.go:52 +0x68

goroutine 8 [syscall]:
os/signal.signal_recv()
    runtime/sigqueue.go:151 +0x38
os/signal.loop()
    os/signal/signal_unix.go:23 +0x1c
created by os/signal.Notify.func1.1
    os/signal/signal.go:151 +0x2c

goroutine 9 [select]:
github.com/zyedidia/tcell/v2.(*tScreen).mainLoop(0xc000190000)
    github.com/zyedidia/tcell/v2@v2.0.9/tscreen.go:1580 +0xe0
created by github.com/zyedidia/tcell/v2.(*tScreen).Init
    github.com/zyedidia/tcell/v2@v2.0.9/tscreen.go:210 +0x6cc

goroutine 10 [IO wait]:
internal/poll.runtime_pollWait(0xffffff8cc35198, 0x72)
    runtime/netpoll.go:302 +0xe0
internal/poll.(*pollDesc).wait(0xc000058e58, 0x72, 0x1)
    internal/poll/fd_poll_runtime.go:83 +0x34
internal/poll.(*pollDesc).waitRead(...)
    internal/poll/fd_poll_runtime.go:88
internal/poll.(*FD).Read(0xc000058e40, {0xc0003fb000, 0x1000, 0x1000})
    internal/poll/fd_unix.go:167 +0x1f0
os.(*File).read(...)
    os/file_posix.go:31
os.(*File).Read(0xc00000c780, {0xc0003fb000, 0x1000, 0x1000})
    os/file.go:119 +0x70
github.com/zyedidia/tcell/v2.(*tScreen).inputLoop(0xc000190000)
    github.com/zyedidia/tcell/v2@v2.0.9/tscreen.go:1633 +0x94
created by github.com/zyedidia/tcell/v2.(*tScreen).Init
    github.com/zyedidia/tcell/v2@v2.0.9/tscreen.go:211 +0x724
mengzhuo commented 2 years ago

It will be fixed after this CL been merged. https://go-review.googlesource.com/c/go/+/409055

phpeterson-usf commented 2 years ago

Thanks. I would be thrilled if this could make the 1.19 release train to support my teaching objectives.

cherrymui commented 2 years ago

Can this reproduce on a real hardware?

For QEMU, I guess it would be possible to configure it to not enable SV57 mode?

phpeterson-usf commented 2 years ago

I believe SV57 is enabled in the 5.18 kernel which is distributed with Debian/riscv64. I have looked unsuccessfully for runtime configuration for that feature. If there's a way to change that in QEMU, I don't know what it is.

I don't have an Unmatched board, and SiFive discontinued them in favor of a pre-release collaboration with Intel.

If it's too late for 1.19, the two possible workarounds I see are:

  1. Downgrade the kernel to pre-SV57
  2. Build Go from source, patching in Ian's change myself
phpeterson-usf commented 2 years ago

FWIW, I built from go1.19rc2 + Ian's change (e9f157e.diff) and cross-compiled micro to avoid bootstrapping Go on riscv64. The resulting micro binary seems to work fine on my Debian/riscv64 guest.

mengzhuo commented 2 years ago

Can this reproduce on a real hardware?

For QEMU, I guess it would be possible to configure it to not enable SV57 mode?

Most of existed real hardwares (that I can get) using sv39

Qemu configured with -cpu=sifive-u54 to disable SV57.

yuzibo commented 2 years ago

Hi, I just have Unmatched boards by hand and I tested it easily:

vimer@unmatched:~/go/proj/bin$ uname -a
Linux unmatched 5.18.0-2-riscv64 #1 SMP Debian 5.18.5-1 (2022-06-16) riscv64 GNU/Linux
vimer@unmatched:~/go/proj/bin$ go env
GO111MODULE=""
GOARCH="riscv64"
GOBIN=""
GOCACHE="/home/vimer/.cache/go-build"
GOENV="/home/vimer/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="riscv64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/vimer/go/proj/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/vimer/go/proj"
GOPRIVATE=""
GOPROXY="https://proxy.golang.com.cn"
GOROOT="/home/vimer/go/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/vimer/go/go/pkg/tool/linux_riscv64"
GOVCS=""
GOVERSION="go1.18.4"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2096220389=/tmp/go-build -gno-record-gcc-switches"

vimer@unmatched:~/go/proj/bin$ ./micro --version
Version: 2.0.11-dev.100
Commit hash: 225927b9
Compiled on July 29, 2022

vimer@unmatched:~/go/proj/bin$ ip a | ./micro

The micro editor runs fine in my view also. Please let me know if there are any issues.

gopherbot commented 2 years ago

Change https://go.dev/cl/409055 mentions this issue: runtime: support riscv64 SV57 mode

4a6f656c commented 2 years ago

This should be fixed via https://go.dev/cl/409055, closing.