cmd/compile: fuzzing triggers various crashes or internal compiler errors on tip

thepudds commented 3 years ago

What version of Go are you using (`go version`)?

$ go version
go version devel go1.18-640a49b Sat Oct 16 16:27:40 2021 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes, tip.

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/thepudds/.cache/go-build"
GOENV="/home/thepudds/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/thepudds/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/thepudds/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/thepudds/sdk/gotip"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/thepudds/sdk/gotip/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="devel go1.18-640a49b Sat Oct 16 16:27:40 2021 +0000"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="0"
GOMOD="/home/thepudds/patch-md-2nd-cl/fuzzgc/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1248123863=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Fuzz cmd/compile using dvyukov/go-fuzz. (This is partially a follow-up to #39634, but there I was just fuzzing the type checker, whereas this was fuzzing the full cmd/compile).

Most of the fuzzing happened on an earlier commit, but roughly half of the original crashers that were initially found no longer reproduce on tip, or seem to be tracked elsewhere (e.g., https://github.com/golang/go/issues/47631#issuecomment-945007646).

What did you see?

Here are 4 crashers that still reproduce on latest tip with go tool compile:

Crash 1: goroutine stack exceeds 1000000000-byte limit

In types2.(*unifier).nifyEq (via nifyEq -> nify -> nifyEq -> nify -> ...)

package n

func O[T any](l, t func() T, f T) { O(t, func() g {}, func() {}) }

Crash 2: internal compiler error: label missing tag

In escape.(*escape).stmt:

package a

func A() {
_:
_:
}

Crash 3: internal compiler error: bvset: index 6 is out of bounds with length 6

In bitvec.BitVec.Set (via typebits.Set and liveness.WriteFuncMap):

package i

type w []w

func (w) a(_ [2 << 40]w)

Crash 4: internal compiler error: bad type

In types.typePos (via types.CalcSize):

package p

type F [][8][2][880][80000][80000]string

(FYI @mdempsky -- this is a belated follow-up from our conversation from a while ago).

robpike commented 3 years ago

Although it's good to find bugs, four decades of using this technique to find bugs in compilers has failed to find much of use in my experience. Others may well feel differently, but most of the problematic generated programs are utter garbage and as long as the compiler fails to compile them, I'm happy. By the time the C fuzzer was generating arrays of functions of volatile unions of registers, I'd lost interest.

That said (and believed), with all the recent change in the compiler it's probably worth running the fuzzer. I would argue though that fixing problems from clearly ridiculous programs is not worth anyone's time. Yet some of these do look easy to fix. The last two are clearly just missed overflows in some form, while the first is just the latest iteration of the problems handling recursive types, a perennial landmine.

One service the reporter can do is to try to whittle the example down to a more compact expression of the problem. These ones win by that metric, so thank you.

findleyr commented 3 years ago

Crash 1 looks like #49019, which we're working on. CC @griesemer

Thank you for the diligent report. It will be interesting to continue fuzzing as we approach the release. Hopefully as we fix the known bugs, crashes that fuzzing finds will become increasingly unrealistic.

thepudds commented 2 years ago

Crash 1 looks like #49019, which we're working on. CC @griesemer

Hi @findleyr, did you maybe mean #48974? (The issue number you mentioned is this issue here).

If so, I see #48974 is closed as of two days ago for the type checker (with a follow-on issue for cmd/compile), but FWIW, I still see Crash 1 from above still happening in the same way on tip as of early this AM with 8b9c0d1, which still hits goroutine stack exceeds 1000000000-byte limit with the same stack as reported above (types2.(*unifier).nifyEq -> nify -> nifyEq -> nify -> ...).

Also, Crash 1 from above might be the same as Crash 14 I reported in #39634 (also found via fuzzing):

Crash 14 from https://github.com/golang/go/issues/39634#issuecomment-646392213: runtime: goroutine stack exceeds 1000000000-byte limit

In types2.(*unifier).nifyEq -> nify -> nifyEq -> nify -> ...:

package main

func F[T *any](func(T) T) { F(func(T) t {}) }

The stack for Crash 14 is now different compared to when it was reported in June 2020, which is presumably expected due evolution since that early prototype. Also, I updated that older crash to the new syntax.

findleyr commented 2 years ago

Hi @findleyr, did you maybe mean #48974? (The issue number you mentioned is this issue here).

:facepalm: sorry, no I meant #48619, which is still open.

golang / go