search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.06k stars 3.8k forks source link

kv/kvserver: TestStoreRangeMergeWithData failed #126469

Closed cockroach-teamcity closed 3 months ago

cockroach-teamcity commented 3 months ago

kv/kvserver.TestStoreRangeMergeWithData failed on master @ d13253527955eaa2da09394b8a2729627ab25c48:

Fatal error:

panic: test timed out after 1h16m37s
running tests:
    TestStoreRangeMergeWithData (1h16m37s)
    TestStoreRangeMergeWithData/retries=3 (1h16m34s)

Stack: 

goroutine 978307 [running]:
testing.(*M).startAlarm.func1()
    GOROOT/src/testing/testing.go:2366 +0x265
created by time.goFunc
    GOROOT/src/time/sleep.go:177 +0x45
Log preceding fatal error

``` golang.org/x/sync/errgroup.(*Group).Wait(0xc005bef5c0) golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:56 +0x38 github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.Wait({0xc005bef5c0?, {0xe6c3320?, 0xc0059fd810?}}) github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:144 +0x65 github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.muxRangeFeed({0xe6c3358, 0xc00496ec90}, {0x1, 0x1, 0x0, 0x0, 0x0, {0x0, 0x0, 0x0, ...}}, ...) github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_mux_rangefeed.go:100 +0x8ef github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeedSpans(0xc004c84608, {0xe6c3320, 0xc0059fd7c0}, {0xc005bef540, 0x1, 0x1}, 0xc006073f80, {0xc001aec5e0, 0x2, 0x2}) github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:227 +0x577 github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed(0xc004c84608, {0xe6c3320, 0xc0059fd7c0}, {0xc0066455c0, 0x1, 0xc0070da240?}, {0xc005cd5ad0?, 0x44825b8?}, 0xc006073f80, {0xc001aec5e0, ...}) github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:162 +0x2e5 github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*dbAdapter).RangeFeed(0xc002f3c5e8, {0xe6c3320, 0xc0059fd7c0}, {0xc0066455c0, 0x1, 0x1}, {0xd9e?, 0x44825b8?}, 0xc006073f80, {0xc001aec5e0, ...}) github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/db_adapter.go:80 +0xed github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).run.func1({0xe6c3320, 0xc0059fd7c0}) github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:365 +0x390 github.com/cockroachdb/cockroach/pkg/util/ctxgroup.GoAndWait.Group.GoCtx.func1() github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x47 golang.org/x/sync/errgroup.(*Group).Go.func1() golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x92 created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 3445 golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x125 goroutine 3500 [select]: github.com/cockroachdb/cockroach/pkg/server.(*serverController).start.func1({0xe6c3358, 0xc004b0b860}) github.com/cockroachdb/cockroach/pkg/server/server_controller.go:215 +0x3b3 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2() github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:485 +0x263 created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 1903 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:476 +0x69d goroutine 3648 [chan receive, 76 minutes]: github.com/cockroachdb/cockroach/pkg/util/netutil.MakeTCPServer.func1({0xc005749008?, 0xc001978360?}) github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:126 +0x65 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2() github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:485 +0x263 created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 1903 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:476 +0x69d goroutine 3732 [select, 76 minutes]: github.com/cockroachdb/cockroach/pkg/kv/kvserver/idalloc.(*Allocator).start.func1({0xe6c3358, 0xc001ede660}) github.com/cockroachdb/cockroach/pkg/kv/kvserver/idalloc/id_alloc.go:151 +0xaf0 github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2() github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:485 +0x263 created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 1903 github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:476 +0x69d goroutine 3731 [select, 76 minutes]: github.com/cockroachdb/cockroach/pkg/server.(*testServer).Activate.func2() github.com/cockroachdb/cockroach/pkg/server/testserver.go:840 +0x1b7 created by github.com/cockroachdb/cockroach/pkg/server.(*testServer).Activate in goroutine 1903 github.com/cockroachdb/cockroach/pkg/server/testserver.go:837 +0x2eb ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #126249 kv/kvserver: TestStoreRangeMergeWithData failed [C-bug C-test-failure O-robot P-3 T-kv]

/cc @cockroachdb/kv

This test on roachdash | Improve this report!

Jira issue: CRDB-39936

arulajmani commented 3 months ago

Duplicate of https://github.com/cockroachdb/cockroach/issues/126249. Not sure why it created a new issue -- maybe because that one was missing a branch-master? Either way, closing.