insights: v23.1.13: nil quantile.Stream in withFingerprintLatencySummary

[cockroach-sentry]

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: https://cockroach-labs.sentry.io/issues/5378038262/?referrer=webhooks_plugin

Panic Message:

panic.go:884: runtime error: invalid memory address or nil pointer dereference
(1) attached stack trace
  -- stack trace:
  | runtime.gopanic
  |     GOROOT/src/runtime/panic.go:884
  | runtime.panicmem
  |     GOROOT/src/runtime/panic.go:260
  | runtime.sigpanic
  |     GOROOT/src/runtime/signal_unix.go:839
  | github.com/cockroachdb/cockroach/pkg/util/quantile.(*Stream).ByteSize
  |     github.com/cockroachdb/cockroach/pkg/util/quantile/stream.go:230
  | github.com/cockroachdb/cockroach/pkg/sql/sqlstats/insights.(*anomalyDetector).withFingerprintLatencySummary
  |     github.com/cockroachdb/cockroach/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/detector.go:135
  | github.com/cockroachdb/cockroach/pkg/sql/sqlstats/insights.(*anomalyDetector).isSlow
  |     github.com/cockroachdb/cockroach/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/detector.go:82
  | github.com/cockroachdb/cockroach/pkg/sql/sqlstats/insights.(*compositeDetector).isSlow
  |     github.com/cockroachdb/cockroach/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/detector.go:50
  | github.com/cockroachdb/cockroach/pkg/sql/sqlstats/insights.(*lockingRegistry).ObserveTransaction
  |     github.com/cockroachdb/cockroach/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/registry.go:103
  | github.com/cockroachdb/cockroach/pkg/sql/sqlstats/insights.(*concurrentBufferIngester).ingest
  |     github.com/cockroachdb/cockroach/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/ingester.go:108
  | github.com/cockroachdb/cockroach/pkg/sql/sqlstats/insights.(*concurrentBufferIngester).Start.func1
  |     github.com/cockroachdb/cockroach/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/ingester.go:72
  | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
  |     github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:470
  | runtime.goexit
  |     GOROOT/src/runtime/asm_amd64.s:1594
Wraps: (2) runtime error: invalid memory address or nil pointer dereference
Error types: (1) *withstack.withStack (2) runtime.errorString
-- report composition:
runtime.errorString
panic.go:884: *withstack.withStack (top exception)

Stacktrace (expand for inline code snippets):

[GOROOT/src/runtime/asm_amd64.s#L1593-L1595](GOROOT/src/runtime/asm_amd64.s#L1593-L1595) [https://github.com/cockroachdb/cockroach/blob/8d065df26e25a762a1abbbf2b1e475456b341a00/pkg/util/stop/stopper.go#L469-L471](https://github.com/cockroachdb/cockroach/blob/8d065df26e25a762a1abbbf2b1e475456b341a00/pkg/util/stop/stopper.go#L469-L471) [https://github.com/cockroachdb/cockroach/blob/8d065df26e25a762a1abbbf2b1e475456b341a00/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/ingester.go#L71-L73](https://github.com/cockroachdb/cockroach/blob/8d065df26e25a762a1abbbf2b1e475456b341a00/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/ingester.go#L71-L73) [https://github.com/cockroachdb/cockroach/blob/8d065df26e25a762a1abbbf2b1e475456b341a00/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/ingester.go#L107-L109](https://github.com/cockroachdb/cockroach/blob/8d065df26e25a762a1abbbf2b1e475456b341a00/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/ingester.go#L107-L109) [https://github.com/cockroachdb/cockroach/blob/8d065df26e25a762a1abbbf2b1e475456b341a00/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/registry.go#L102-L104](https://github.com/cockroachdb/cockroach/blob/8d065df26e25a762a1abbbf2b1e475456b341a00/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/registry.go#L102-L104) [https://github.com/cockroachdb/cockroach/blob/8d065df26e25a762a1abbbf2b1e475456b341a00/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/detector.go#L49-L51](https://github.com/cockroachdb/cockroach/blob/8d065df26e25a762a1abbbf2b1e475456b341a00/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/detector.go#L49-L51) [https://github.com/cockroachdb/cockroach/blob/8d065df26e25a762a1abbbf2b1e475456b341a00/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/detector.go#L81-L83](https://github.com/cockroachdb/cockroach/blob/8d065df26e25a762a1abbbf2b1e475456b341a00/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/detector.go#L81-L83) [https://github.com/cockroachdb/cockroach/blob/8d065df26e25a762a1abbbf2b1e475456b341a00/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/detector.go#L134-L136](https://github.com/cockroachdb/cockroach/blob/8d065df26e25a762a1abbbf2b1e475456b341a00/pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/detector.go#L134-L136) [https://github.com/cockroachdb/cockroach/blob/8d065df26e25a762a1abbbf2b1e475456b341a00/pkg/util/quantile/stream.go#L229-L231](https://github.com/cockroachdb/cockroach/blob/8d065df26e25a762a1abbbf2b1e475456b341a00/pkg/util/quantile/stream.go#L229-L231) [GOROOT/src/runtime/signal_unix.go#L838-L840](GOROOT/src/runtime/signal_unix.go#L838-L840) [GOROOT/src/runtime/panic.go#L259-L261](GOROOT/src/runtime/panic.go#L259-L261) [GOROOT/src/runtime/panic.go#L883-L885](GOROOT/src/runtime/panic.go#L883-L885)

GOROOT/src/runtime/asm_amd64.s in runtime.goexit at line 1594
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 at line 470
pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/ingester.go in pkg/sql/sqlstats/insights.(*concurrentBufferIngester).Start.func1 at line 72
pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/ingester.go in pkg/sql/sqlstats/insights.(*concurrentBufferIngester).ingest at line 108
pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/registry.go in pkg/sql/sqlstats/insights.(*lockingRegistry).ObserveTransaction at line 103
pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/detector.go in pkg/sql/sqlstats/insights.(*compositeDetector).isSlow at line 50
pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/detector.go in pkg/sql/sqlstats/insights.(*anomalyDetector).isSlow at line 82
pkg/sql/sqlstats/insights/pkg/sql/sqlstats/insights/detector.go in pkg/sql/sqlstats/insights.(*anomalyDetector).withFingerprintLatencySummary at line 135
pkg/util/quantile/stream.go in pkg/util/quantile.(*Stream).ByteSize at line 230
GOROOT/src/runtime/signal_unix.go in runtime.sigpanic at line 839
GOROOT/src/runtime/panic.go in runtime.panicmem at line 260
GOROOT/src/runtime/panic.go in runtime.gopanic at line 884

Tags

Tag	Value
Command	start-single-node
Environment	v23.1.13
Go Version	go1.19.13
Platform	linux amd64
Distribution	CCL
Cockroach Release	v23.1.13
Cockroach SHA	8d065df26e25a762a1abbbf2b1e475456b341a00
# of CPUs	6
# of Goroutines	382

Jira issue: CRDB-38874

[yuzefovich]

seems to have the same root cause as #124407

[yuzefovich]

Actually never mind, this seems different from #124407