Open bcmills opened 2 years ago
cc @mknyszek
greplogs -l -e 'found wrong event in p'\''s limiter event slot' --since=2022-09-03
2022-09-06T16:53:53-dc029f2/openbsd-amd64-68
Found new dashboard test flakes for:
#!watchflakes
post <- `found wrong event in p's limiter event slot`
— watchflakes
Found new dashboard test flakes for:
#!watchflakes
post <- `found wrong event in p's limiter event slot`
— watchflakes
Found new dashboard test flakes for:
#!watchflakes
post <- `found wrong event in p's limiter event slot`
— watchflakes
Found new dashboard test flakes for:
#!watchflakes
post <- `found wrong event in p's limiter event slot`
— watchflakes
Found new dashboard test flakes for:
#!watchflakes
post <- `found wrong event in p's limiter event slot`
— watchflakes
Found new dashboard test flakes for:
#!watchflakes
post <- `found wrong event in p's limiter event slot`
— watchflakes
Found new dashboard test flakes for:
#!watchflakes
post <- `found wrong event in p's limiter event slot`
— watchflakes
Found new dashboard test flakes for:
#!watchflakes
post <- `found wrong event in p's limiter event slot`
— watchflakes
I have been regularly seeing fatals with this error on FreeBSD AMD64 running go1.21.7.
See here for the error:
runtime: want=4 got=0
fatal error: limiterEvent.stop: found wrong event in p's limiter event slot
runtime stack:
runtime.throw({0x109ba34?, 0x465772?})
/usr/local/go/src/runtime/panic.go:1077 +0x5c fp=0xc00040fe68 sp=0xc00040fe38 pc=0x439f1c
runtime.(*limiterEvent).stop(0x43684e?, 0x4, 0x4?)
/usr/local/go/src/runtime/mgclimit.go:454 +0x14a fp=0xc00040fe90 sp=0xc00040fe68 pc=0x42022a
runtime.pidleget(0xc00040ff10?)
/usr/local/go/src/runtime/proc.go:6131 +0x125 fp=0xc00040fec8 sp=0xc00040fe90 pc=0x448f65
runtime.pidlegetSpinning(0x1944e78?)
/usr/local/go/src/runtime/proc.go:6149 +0x13 fp=0xc00040fee0 sp=0xc00040fec8 pc=0x448fd3
runtime.wakep()
/usr/local/go/src/runtime/proc.go:2740 +0x65 fp=0xc00040ff10 sp=0xc00040fee0 pc=0x441005
runtime.resetspinning()
/usr/local/go/src/runtime/proc.go:3461 +0x3e fp=0xc00040ff30 sp=0xc00040ff10 pc=0x442c1e
runtime.schedule()
/usr/local/go/src/runtime/proc.go:3595 +0x10f fp=0xc00040ff68 sp=0xc00040ff30 pc=0x44304f
runtime.park_m(0xc00457a340?)
/usr/local/go/src/runtime/proc.go:3740 +0x11f fp=0xc00040ffb0 sp=0xc00040ff68 pc=0x4434ff
runtime.mcall()
/usr/local/go/src/runtime/asm_amd64.s:458 +0x4e fp=0xc00040ffc8 sp=0xc00040ffb0 pc=0x46d10e
goroutine 1 [chan receive, 738 minutes]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000627aa8 sp=0xc000627a88 pc=0x43cd0e
runtime.chanrecv(0xc00010c300, 0x0, 0x1)
/usr/local/go/src/runtime/chan.go:583 +0x3cd fp=0xc000627b20 sp=0xc000627aa8 pc=0x409acd
runtime.chanrecv1(0x0?, 0x0?)
/usr/local/go/src/runtime/chan.go:442 +0x12 fp=0xc000627b48 sp=0xc000627b20 pc=0x4096d2
gitlab/filehosting/server-manager/daemon.(*Daemon).WaitForShutdown(0xc00013aa00)
/builds/filehosting/server-manager/daemon/daemon.go:176 +0x45 fp=0xc000627c48 sp=0xc000627b48 pc=0xe0ea65
gitlab/filehosting/server-manager/cmd.startDaemon(0xc000037700?, {0x104be0f?, 0x4?, 0x104bd27?})
/builds/filehosting/server-manager/cmd/daemon.go:31 +0x2f fp=0xc000627c68 sp=0xc000627c48 pc=0xe2c66f
github.com/spf13/cobra.(*Command).execute(0x1937540, {0xc0004a67c0, 0x2, 0x2})
/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:983 +0xabc fp=0xc000627e08 sp=0xc000627c68 pc=0x5cfe9c
github.com/spf13/cobra.(*Command).ExecuteC(0x1939200)
/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1115 +0x3ff fp=0xc000627ee0 sp=0xc000627e08 pc=0x5d075f
github.com/spf13/cobra.(*Command).Execute(0x0?)
/go/pkg/mod/github.com/spf13/cobra@v1.8.0/command.go:1039 +0x13 fp=0xc000627ef8 sp=0xc000627ee0 pc=0x5d02b3
main.main()
/builds/filehosting/server-manager/server-manager.go:13 +0x1a fp=0xc000627f40 sp=0xc000627ef8 pc=0xe2e75a
runtime.main()
/usr/local/go/src/runtime/proc.go:267 +0x2b2 fp=0xc000627fe0 sp=0xc000627f40 pc=0x43c8b2
(tons more goroutines..)
Is there anything I can do to help debug the issue and help solve it?
Found new dashboard test flakes for:
#!watchflakes
post <- (goos == "plan9" || goos == "openbsd") && `found wrong event in p's limiter event slot`
— watchflakes
Found new dashboard test flakes for:
#!watchflakes
post <- (goos == "plan9" || goos == "openbsd") && `found wrong event in p's limiter event slot`
— watchflakes
Found new dashboard test flakes for:
#!watchflakes
post <- (goos == "plan9" || goos == "openbsd") && `found wrong event in p's limiter event slot`
— watchflakes
Found new dashboard test flakes for:
#!watchflakes
post <- (goos == "plan9" || goos == "openbsd") && `found wrong event in p's limiter event slot`
— watchflakes
greplogs -l -e 'found wrong event in p'\''s limiter event slot' --since=2022-01-01
2022-08-18T17:26:40-52016be/plan9-arm 2022-08-16T20:39:44-046e8b3-e49e876/plan9-arm 2022-08-11T20:13:07-8743198/openbsd-amd64-68 2022-08-10T23:27:37-29b0703/openbsd-amd64-68 2022-08-08T14:10:37-5405df0/plan9-armAttn @golang/runtime; I have no idea what the common thread between these two builders might be. 😅