search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.06k stars 3.8k forks source link

ccl/backupccl: TestClusterRestoreFailCleanup failed #124798

Closed cockroach-teamcity closed 4 months ago

cockroach-teamcity commented 5 months ago

ccl/backupccl.TestClusterRestoreFailCleanup failed on release-24.1.0-rc @ 5e4ca9e26f1a25681de9c944298cfa139c344466:

      github.com/cockroachdb/cockroach/pkg/kv/db.go:1052 +0xec
  github.com/cockroachdb/cockroach/pkg/kv.(*DB).Txn()
      github.com/cockroachdb/cockroach/pkg/kv/db.go:1027 +0x2866
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).adminSplitWithDescriptor()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replica_command.go:553 +0x1477
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*splitQueue).processAttempt()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/split_queue.go:292 +0x1e9
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*splitQueue).processAttemptWithTracing()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/split_queue.go:254 +0x259
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*splitQueue).process()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/split_queue.go:229 +0x8b
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica.func1()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:984 +0x116
  github.com/cockroachdb/cockroach/pkg/util/timeutil.RunWithTimeout()
      github.com/cockroachdb/cockroach/pkg/util/timeutil/timeout.go:29 +0x12d
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processReplica()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:977 +0x65d
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processOneAsyncAndReleaseSem.func1()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:927 +0x15a
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2()
      github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:485 +0x262

Goroutine 23557 (running) created at:
  github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*Server).serveImpl()
      github.com/cockroachdb/cockroach/pkg/sql/pgwire/server.go:989 +0xa4f
  github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*Server).ServeConn()
      github.com/cockroachdb/cockroach/pkg/sql/pgwire/server.go:805 +0xe04
  github.com/cockroachdb/cockroach/pkg/server.(*tenantServerWrapper).serveConn()
      github.com/cockroachdb/cockroach/pkg/server/server_controller_sql.go:191 +0x1f7
  github.com/cockroachdb/cockroach/pkg/server.(*serverController).sqlMux()
      github.com/cockroachdb/cockroach/pkg/server/server_controller_sql.go:95 +0x561
  github.com/cockroachdb/cockroach/pkg/server.(*serverController).sqlMux-fm()
      <autogenerated>:1 +0xc4
  github.com/cockroachdb/cockroach/pkg/server.startServeSQL.func1.1()
      github.com/cockroachdb/cockroach/pkg/server/server_sql.go:1876 +0x28b
  github.com/cockroachdb/cockroach/pkg/util/netutil.(*TCPServer).ServeWith.func1()
      github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:191 +0x1c6
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2()
      github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:485 +0x262

Goroutine 25153 (running) created at:
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx()
      github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:476 +0x69c
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processOneAsyncAndReleaseSem()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:922 +0x513
  github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2()
      github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/queue.go:881 +0x298
  github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2()
      github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:485 +0x262
==================

Parameters:

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/disaster-recovery

This test on roachdash | Improve this report!

Jira issue: CRDB-39077

msbutler commented 4 months ago

same race as https://github.com/cockroachdb/cockroach/issues/124553. Closing as dupe.