Closed cockroach-teamcity closed 1 year ago
@lidorcarmel I'm throwing this your way since you've thought about this a bit. Happy to look into it with you though.
(cc @knz for visibility)
See stack below.
I'm thinking just to put a mutex around serverStateUsingChannels.server
, I'll send a pr tomorrow.
==================
07:39:31 WARNING: DATA RACE
07:39:31 Write at 0x00c003a18828 by goroutine 147262:
07:39:31 github.com/cockroachdb/cockroach/pkg/server.(*channelOrchestrator).startControlledServer.func5()
07:39:31 github.com/cockroachdb/cockroach/pkg/server/server_controller_channel_orchestrator.go:416 +0xa64
07:39:31 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2()
07:39:31 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:470 +0x1f6
07:39:31
07:39:31 Previous read at 0x00c003a18828 by goroutine 147620:
07:39:31 github.com/cockroachdb/cockroach/pkg/server.(*serverStateUsingChannels).getServer()
07:39:31 github.com/cockroachdb/cockroach/pkg/server/server_controller_channel_orchestrator.go:109 +0x4e
07:39:31 github.com/cockroachdb/cockroach/pkg/server.(*serverController).getServer()
07:39:31 github.com/cockroachdb/cockroach/pkg/server/server_controller_accessors.go:28 +0x138
07:39:31 github.com/cockroachdb/cockroach/pkg/server.(*serverController).sqlMux()
07:39:31 github.com/cockroachdb/cockroach/pkg/server/server_controller_sql.go:67 +0x2ac
07:39:31 github.com/cockroachdb/cockroach/pkg/server.(*serverController).sqlMux-fm()
07:39:31 <autogenerated>:1 +0xc7
07:39:31 github.com/cockroachdb/cockroach/pkg/server.startServeSQL.func1.1()
07:39:31 github.com/cockroachdb/cockroach/pkg/server/server_sql.go:1766 +0x2eb
07:39:31 github.com/cockroachdb/cockroach/pkg/util/netutil.(*TCPServer).ServeWith.func1()
07:39:31 github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:188 +0x111
07:39:31 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2()
07:39:31 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:470 +0x1f6
07:39:31
07:39:31 Goroutine 147262 (running) created at:
07:39:31 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx()
07:39:31 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:461 +0x619
07:39:31 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask()
07:39:31 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:332 +0xf4c
07:39:31 github.com/cockroachdb/cockroach/pkg/server.(*channelOrchestrator).startControlledServer()
07:39:31 github.com/cockroachdb/cockroach/pkg/server/server_controller_channel_orchestrator.go:292 +0x29
07:39:31 github.com/cockroachdb/cockroach/pkg/server.(*serverController).createServerEntryLocked()
07:39:31 github.com/cockroachdb/cockroach/pkg/server/server_controller_orchestration.go:173 +0x2b0
07:39:31 github.com/cockroachdb/cockroach/pkg/server.(*serverController).scanTenantsForRunnableServices()
07:39:31 github.com/cockroachdb/cockroach/pkg/server/server_controller_orchestration.go:134 +0x538
07:39:31 github.com/cockroachdb/cockroach/pkg/server.(*serverController).start.func1()
07:39:31 github.com/cockroachdb/cockroach/pkg/server/server_controller_orchestration.go:60 +0x21a
07:39:31 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2()
07:39:31 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:470 +0x1f6
07:39:31
07:39:31 Goroutine 147620 (running) created at:
07:39:31 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx()
07:39:31 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:461 +0x619
07:39:31 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask()
07:39:31 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:332 +0x404
07:39:31 github.com/cockroachdb/cockroach/pkg/util/netutil.(*TCPServer).ServeWith()
07:39:31 github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:185 +0x36
07:39:31 github.com/cockroachdb/cockroach/pkg/server.startServeSQL.func1()
07:39:31 github.com/cockroachdb/cockroach/pkg/server/server_sql.go:1756 +0x17b
07:39:31 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2()
07:39:31 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:470 +0x1f6
07:39:31 ==================
ccl/streamingccl/streamingest.TestDataDriven failed with artifacts on release-23.1 @ f6c68f6626497c43f2e5bef6f7e189b8792cfefb:
Parameters:
TAGS=bazel,gss,race
Help
See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)
/cc @cockroachdb/disaster-recovery
This test on roachdash | Improve this report!
Jira issue: CRDB-30261