search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.18k stars 3.82k forks source link

kv/kvserver: TestTxnReadWithinUncertaintyIntervalAfterLeaseTransfer failed #131003

Open cockroach-teamcity opened 2 months ago

cockroach-teamcity commented 2 months ago

kv/kvserver.TestTxnReadWithinUncertaintyIntervalAfterLeaseTransfer failed on release-24.1.5-rc @ 08092610980d10494d90726d6aaee007d4d23c1e:

=== RUN   TestTxnReadWithinUncertaintyIntervalAfterLeaseTransfer
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestTxnReadWithinUncertaintyIntervalAfterLeaseTransfer3798687909
    test_log_scope.go:81: use -show-logs to present logs inline
    client_replica_test.go:867: 
            Error Trace:    github.com/cockroachdb/cockroach/pkg/kv/kvserver_test/pkg/kv/kvserver/client_replica_test.go:867
            Error:          Should be true
            Test:           TestTxnReadWithinUncertaintyIntervalAfterLeaseTransfer
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestTxnReadWithinUncertaintyIntervalAfterLeaseTransfer3798687909
--- FAIL: TestTxnReadWithinUncertaintyIntervalAfterLeaseTransfer (0.63s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/kv

This test on roachdash | Improve this report!

Jira issue: CRDB-42332

tbg commented 2 months ago

This failed here

/pkg/kv/kvserver/client_replica_test.go#L867

    require.True(t, txn.ObservedTimestamps[0].Timestamp.ToTimestamp().Less(writeTs))

where as the sibling failure^1 (from 24.1) was here:

/pkg/kv/kvserver/client_replica_test.go#L865

    require.True(t, txn.ReadTimestamp.Less(writeTs))

Now that we have two failures here, and no failures on 24.2+, it stands to reason that this is something specific to the 24.1 branch. I'll kick the tires on this again.

tbg commented 2 months ago 
    Testing //pkg/kv/kvserver:kvserver_test (run 53835 of 200000); 1s linux-sandbox

I don't think this one's going to fail. It might need exacerbating circumstances.

Looking at git log --oneline --topo-order --no-merges -- ./pkg/kv/kvserver/ 08092610980d10494d90726d6aaee007d4d23c1e, the most recent commit that could sort of pattern match onto what we're seeing here is fa5d7f718bede516966b93abf3eff1560cfd9420 which was merged two months ago^1. Yet, we've seen this fail in CI twice on this branch in a few days.