Open bradfitz opened 2 months ago
Similar Issues
(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)
It says the following in runtime1.go
// gotraceback returns the current traceback settings.
//
// If level is 0, suppress all tracebacks.
// If level is 1, show tracebacks, but exclude runtime frames.
// If level is 2, show tracebacks including runtime frames.
// If all is set, print all goroutine stacks. Otherwise, print just the current goroutine.
// If crash is set, crash (core dump, etc) after tracebacking.
Go version
go version go1.22.4 linux/amd64
What did you do?
What did you see happen?
... we hit the race on the map in prod after an hour and 💣
The Go runtime then did its
fatal
:But then we got 3 GB of GOTRACEBACK stacks from the Go runtime for 1M+ stacks, about 3 GB more than we needed to find the bug, overwhelming our logging system in the process, causing a secondary outage.
GOTRACEBACK docs say:
And arguably a map race is internal to the run-time insofar as maps are implemented in the runtime.
But the Go runtime know it's the user's fault; note the "but is used when user code is expected to be at fault for the failure" bit here:
What did you expect to see?
I'd expect
fatalthrow(throwTypeUser)
to be treated as a normal panic withGOTRACEBACK=single
respected.Currently it's ignored:
(At least
none
works, but that's too quiet, hiding the actual problem).