Sentry: hlc.go:421: log.Fatal: wall time 1709553902972723299 is not allowed to be greater than upper bound of 1709553902970633980. (1) attached stack trace -- stack trace: | github.com/cockroachdb... #119859
GOROOT/src/runtime/asm_amd64.s in runtime.goexit at line 1594
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 at line 470
pkg/server/server_controller_orchestration.go in pkg/server.(*serverController).start.func1 at line 60
pkg/server/server_controller_orchestration.go in pkg/server.(*serverController).scanTenantsForRunnableServices at line 106
pkg/server/server_controller_orchestration.go in pkg/server.(*serverController).getExpectedRunningTenants at line 195
pkg/sql/internal.go in pkg/sql.(*InternalExecutor).QueryIterator at line 714
pkg/sql/internal.go in pkg/sql.(*InternalExecutor).QueryIteratorEx at line 728
pkg/sql/internal.go in pkg/sql.(*InternalExecutor).execInternal at line 1020
pkg/sql/internal.go in pkg/sql.(*InternalExecutor).runWithEx at line 169
pkg/sql/internal.go in pkg/sql.(*InternalExecutor).initConnEx at line 239
pkg/sql/exec_util.go in pkg/sql.(*ExecutorConfig).GenerateID at line 1490
pkg/util/hlc/pkg/util/hlc/hlc.go in pkg/util/hlc.(*Clock).Now at line 389
pkg/util/hlc/pkg/util/hlc/hlc.go in pkg/util/hlc.(*Clock).NowAsClockTimestamp at line 412
pkg/util/hlc/pkg/util/hlc/hlc.go in pkg/util/hlc.(*Clock).enforceWallTimeWithinBoundLocked at line 421
This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.
Sentry Link: https://cockroach-labs.sentry.io/issues/5035239988/?referrer=webhooks_plugin
Panic Message:
Stacktrace (expand for inline code snippets):
[GOROOT/src/runtime/asm_amd64.s#L1593-L1595](GOROOT/src/runtime/asm_amd64.s#L1593-L1595) [https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/util/stop/stopper.go#L469-L471](https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/util/stop/stopper.go#L469-L471) [https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/server/server_controller_orchestration.go#L59-L61](https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/server/server_controller_orchestration.go#L59-L61) [https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/server/server_controller_orchestration.go#L105-L107](https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/server/server_controller_orchestration.go#L105-L107) [https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/server/server_controller_orchestration.go#L194-L196](https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/server/server_controller_orchestration.go#L194-L196) [https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/sql/internal.go#L713-L715](https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/sql/internal.go#L713-L715) [https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/sql/internal.go#L727-L729](https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/sql/internal.go#L727-L729) [https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/sql/internal.go#L1019-L1021](https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/sql/internal.go#L1019-L1021) [https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/sql/internal.go#L168-L170](https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/sql/internal.go#L168-L170) [https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/sql/internal.go#L238-L240](https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/sql/internal.go#L238-L240) [https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/sql/exec_util.go#L1489-L1491](https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/sql/exec_util.go#L1489-L1491) [https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/util/hlc/pkg/util/hlc/hlc.go#L388-L390](https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/util/hlc/pkg/util/hlc/hlc.go#L388-L390) [https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/util/hlc/pkg/util/hlc/hlc.go#L411-L413](https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/util/hlc/pkg/util/hlc/hlc.go#L411-L413) [https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/util/hlc/pkg/util/hlc/hlc.go#L420-L422](https://github.com/cockroachdb/cockroach/blob/62ad175cbc0e9041d993ea1cc911e07d279671f5/pkg/util/hlc/pkg/util/hlc/hlc.go#L420-L422)Tags
Jira issue: CRDB-36356