kv/kvserver: TestStoreRangeMergeSlowUnabandonedFollower_NoSplit failed

[cockroach-teamcity]

kv/kvserver.TestStoreRangeMergeSlowUnabandonedFollower_NoSplit failed with artifacts on release-23.1 @ d17717542620cb88090fb0377e4e08d6a816f1f7:

          | github.com/cockroachdb/cockroach/pkg/testutils/testcluster.(*TestCluster).TransferRangeLeaseOrFatal
          |     github.com/cockroachdb/cockroach/pkg/testutils/testcluster/testcluster.go:1068
          | github.com/cockroachdb/cockroach/pkg/kv/kvserver_test.TestStoreRangeMergeSlowUnabandonedFollower_NoSplit
          |     github.com/cockroachdb/cockroach/pkg/kv/kvserver_test/pkg/kv/kvserver/client_merge_test.go:2707
          | testing.tRunner
          |     GOROOT/src/testing/testing.go:1446
          | runtime.goexit
          |     GOROOT/src/runtime/asm_amd64.s:1594
        Wraps: (2) /Table/Max: transfer lease unexpected error
        Wraps: (3) forced error mark
          | "lease transfer rejected because the target may need a snapshot"
          | github.com/cockroachdb/errors/withstack/*withstack.withStack::
        Wraps: (4)
          | (opaque error wrapper)
          | type name: github.com/cockroachdb/errors/withstack/*withstack.withStack
          | reportable 0:
          |
          | github.com/cockroachdb/cockroach/pkg/kv/kvserver.NewLeaseTransferRejectedBecauseTargetMayNeedSnapshotError
          |     github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replica_range_lease.go:1100
          | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*replicaProposer).rejectProposalWithLeaseTransferRejectedLocked
          |     github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replica_proposal_buf.go:1289
          | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*propBuf).maybeRejectUnsafeProposalLocked
          |     github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replica_proposal_buf.go:710
          | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*propBuf).FlushLockedWithRaftGroup
          |     github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replica_proposal_buf.go:428
          | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReadyRaftMuLocked.func2
          |     github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replica_raft.go:779
          | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).withRaftGroupLocked.func1
          |     github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replica_raft.go:2009
          | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).withRaftGroupLocked
          |     github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replica_raft.go:2010
          | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReadyRaftMuLocked
          |     github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replica_raft.go:776
          | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Replica).handleRaftReady
          |     github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/replica_raft.go:732
          | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).processReady
          |     github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/store_raft.go:646
          | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftSchedulerShard).worker
          |     github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/scheduler.go:395
          | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).Start.func2
          |     github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/scheduler.go:302
          | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
          |     github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:470
          | runtime.goexit
          |     GOROOT/src/runtime/asm_amd64.s:1594
        Wraps: (5) refusing to transfer lease to (n3,s3):3 because target may need a Raft snapshot: local replica not raft leader
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *markers.withMark (4) *errbase.opaqueWrapper (5) *errutil.leafError
    panic.go:522: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/09357cecfdbbab5926b4b055936a9b62/logTestStoreRangeMergeSlowUnabandonedFollower_NoSplit2249298659
--- FAIL: TestStoreRangeMergeSlowUnabandonedFollower_NoSplit (4.54s)

Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)

/cc @cockroachdb/kv

This test on roachdash | Improve this report!

Jira issue: CRDB-42291

[tbg]

This looks like a repeat of https://github.com/cockroachdb/cockroach/issues/127431, for which @nvanbenschoten has a draft PR out^1.

[github-actions[bot]]

We have marked this test failure issue as stale because it has been inactive for 1 month. If this failure is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 5 days to keep the test failure queue tidy.