Closed cockroach-teamcity closed 3 months ago
kv/kvserver.TestStoreRangeMergeWithData failed on master @ d13253527955eaa2da09394b8a2729627ab25c48:
Fatal error:
panic: test timed out after 1h16m37s running tests: TestStoreRangeMergeWithData (1h16m37s) TestStoreRangeMergeWithData/retries=3 (1h16m34s)
Stack:
goroutine 978307 [running]: testing.(*M).startAlarm.func1() GOROOT/src/testing/testing.go:2366 +0x265 created by time.goFunc GOROOT/src/time/sleep.go:177 +0x45
``` golang.org/x/sync/errgroup.(*Group).Wait(0xc005bef5c0) golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:56 +0x38 github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.Wait({0xc005bef5c0?, {0xe6c3320?, 0xc0059fd810?}}) github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:144 +0x65 github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.muxRangeFeed({0xe6c3358, 0xc00496ec90}, {0x1, 0x1, 0x0, 0x0, 0x0, {0x0, 0x0, 0x0, ...}}, ...) github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_mux_rangefeed.go:100 +0x8ef github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeedSpans(0xc004c84608, {0xe6c3320, 0xc0059fd7c0}, {0xc005bef540, 0x1, 0x1}, 0xc006073f80, {0xc001aec5e0, 0x2, 0x2}) github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:227 +0x577 github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed(0xc004c84608, {0xe6c3320, 0xc0059fd7c0}, {0xc0066455c0, 0x1, 0xc0070da240?}, {0xc005cd5ad0?, 0x44825b8?}, 0xc006073f80, {0xc001aec5e0, ...}) github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:162 +0x2e5 github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*dbAdapter).RangeFeed(0xc002f3c5e8, {0xe6c3320, 0xc0059fd7c0}, {0xc0066455c0, 0x1, 0x1}, {0xd9e?, 0x44825b8?}, 0xc006073f80, {0xc001aec5e0, ...}) github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/db_adapter.go:80 +0xed github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).run.func1({0xe6c3320, 0xc0059fd7c0}) github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:365 +0x390 github.com/cockroachdb/cockroach/pkg/util/ctxgroup.GoAndWait.Group.GoCtx.func1() github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x47 golang.org/x/sync/errgroup.(*Group).Go.func1() golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x92 created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 3445 golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x125 goroutine 3500 [select]: github.com/cockroachdb/cockroach/pkg/server.(*serverController).start.func1({0xe6c3358, 0xc004b0b860}) github.com/cockroachdb/cockroach/pkg/server/server_controller.go:215 +0x3b3 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2() github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:485 +0x263 created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 1903 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:476 +0x69d goroutine 3648 [chan receive, 76 minutes]: github.com/cockroachdb/cockroach/pkg/util/netutil.MakeTCPServer.func1({0xc005749008?, 0xc001978360?}) github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:126 +0x65 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2() github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:485 +0x263 created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 1903 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:476 +0x69d goroutine 3732 [select, 76 minutes]: github.com/cockroachdb/cockroach/pkg/kv/kvserver/idalloc.(*Allocator).start.func1({0xe6c3358, 0xc001ede660}) github.com/cockroachdb/cockroach/pkg/kv/kvserver/idalloc/id_alloc.go:151 +0xaf0 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2() github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:485 +0x263 created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 1903 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:476 +0x69d goroutine 3731 [select, 76 minutes]: github.com/cockroachdb/cockroach/pkg/server.(*testServer).Activate.func2() github.com/cockroachdb/cockroach/pkg/server/testserver.go:840 +0x1b7 created by github.com/cockroachdb/cockroach/pkg/server.(*testServer).Activate in goroutine 1903 github.com/cockroachdb/cockroach/pkg/server/testserver.go:837 +0x2eb ```
Parameters:
attempt=1
race=true
run=1
shard=35
See also: How To Investigate a Go Test Failure (internal)
- #126249 kv/kvserver: TestStoreRangeMergeWithData failed [C-bug C-test-failure O-robot P-3 T-kv]
/cc @cockroachdb/kv
This test on roachdash | Improve this report!
Jira issue: CRDB-39936
Duplicate of https://github.com/cockroachdb/cockroach/issues/126249. Not sure why it created a new issue -- maybe because that one was missing a branch-master? Either way, closing.
branch-master
kv/kvserver.TestStoreRangeMergeWithData failed on master @ d13253527955eaa2da09394b8a2729627ab25c48:
Fatal error:
Stack:
Log preceding fatal error
``` golang.org/x/sync/errgroup.(*Group).Wait(0xc005bef5c0) golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:56 +0x38 github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.Wait({0xc005bef5c0?, {0xe6c3320?, 0xc0059fd810?}}) github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:144 +0x65 github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.muxRangeFeed({0xe6c3358, 0xc00496ec90}, {0x1, 0x1, 0x0, 0x0, 0x0, {0x0, 0x0, 0x0, ...}}, ...) github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_mux_rangefeed.go:100 +0x8ef github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeedSpans(0xc004c84608, {0xe6c3320, 0xc0059fd7c0}, {0xc005bef540, 0x1, 0x1}, 0xc006073f80, {0xc001aec5e0, 0x2, 0x2}) github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:227 +0x577 github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed(0xc004c84608, {0xe6c3320, 0xc0059fd7c0}, {0xc0066455c0, 0x1, 0xc0070da240?}, {0xc005cd5ad0?, 0x44825b8?}, 0xc006073f80, {0xc001aec5e0, ...}) github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:162 +0x2e5 github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*dbAdapter).RangeFeed(0xc002f3c5e8, {0xe6c3320, 0xc0059fd7c0}, {0xc0066455c0, 0x1, 0x1}, {0xd9e?, 0x44825b8?}, 0xc006073f80, {0xc001aec5e0, ...}) github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/db_adapter.go:80 +0xed github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).run.func1({0xe6c3320, 0xc0059fd7c0}) github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:365 +0x390 github.com/cockroachdb/cockroach/pkg/util/ctxgroup.GoAndWait.Group.GoCtx.func1() github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x47 golang.org/x/sync/errgroup.(*Group).Go.func1() golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x92 created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 3445 golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x125 goroutine 3500 [select]: github.com/cockroachdb/cockroach/pkg/server.(*serverController).start.func1({0xe6c3358, 0xc004b0b860}) github.com/cockroachdb/cockroach/pkg/server/server_controller.go:215 +0x3b3 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2() github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:485 +0x263 created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 1903 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:476 +0x69d goroutine 3648 [chan receive, 76 minutes]: github.com/cockroachdb/cockroach/pkg/util/netutil.MakeTCPServer.func1({0xc005749008?, 0xc001978360?}) github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:126 +0x65 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2() github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:485 +0x263 created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 1903 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:476 +0x69d goroutine 3732 [select, 76 minutes]: github.com/cockroachdb/cockroach/pkg/kv/kvserver/idalloc.(*Allocator).start.func1({0xe6c3358, 0xc001ede660}) github.com/cockroachdb/cockroach/pkg/kv/kvserver/idalloc/id_alloc.go:151 +0xaf0 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2() github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:485 +0x263 created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 1903 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:476 +0x69d goroutine 3731 [select, 76 minutes]: github.com/cockroachdb/cockroach/pkg/server.(*testServer).Activate.func2() github.com/cockroachdb/cockroach/pkg/server/testserver.go:840 +0x1b7 created by github.com/cockroachdb/cockroach/pkg/server.(*testServer).Activate in goroutine 1903 github.com/cockroachdb/cockroach/pkg/server/testserver.go:837 +0x2eb ```
Parameters:
attempt=1
race=true
run=1
shard=35
Help
See also: How To Investigate a Go Test Failure (internal)
Same failure on other branches
- #126249 kv/kvserver: TestStoreRangeMergeWithData failed [C-bug C-test-failure O-robot P-3 T-kv]
/cc @cockroachdb/kv
This test on roachdash | Improve this report!
Jira issue: CRDB-39936