cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.19k stars 3.82k forks source link

kv/kvserver/reports: TestCriticalLocalitiesReportIntegration failed #134948

Open cockroach-teamcity opened 2 weeks ago

cockroach-teamcity commented 2 weeks ago

kv/kvserver/reports.TestCriticalLocalitiesReportIntegration failed on release-24.2 @ be69012d37da70d0ada6eb4cf2b008d02ef16ffd:

=== RUN   TestCriticalLocalitiesReportIntegration
    test_log_scope.go:165: test logs captured to: outputs.zip/logTestCriticalLocalitiesReportIntegration2099494240
    test_log_scope.go:76: use -show-logs to present logs inline
    reporter_test.go:208: condition failed to evaluate within 3m45s: no report yet
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestCriticalLocalitiesReportIntegration2099494240
--- FAIL: TestCriticalLocalitiesReportIntegration (380.46s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #132157 kv/kvserver/reports: TestCriticalLocalitiesReportIntegration failed [C-bug C-test-failure O-24.3-scale-testing O-robot O-testcluster P-3 T-kv branch-release-24.1.6-rc branch-release-24.3]

/cc @cockroachdb/kv

This test on roachdash | Improve this report!

Jira issue: CRDB-44310

tbg commented 2 weeks ago

This might just be an overloaded cluster. This test has failed various times in the past. This was under testrace, too, and the logs look mighty unhappy

W241112 11:03:28.389772 9290 kv/kvclient/kvcoord/dist_sender.go:2751 ⋮ [T1,Vsystem,n3,ts-poll] 1127 slow replica RPC: have been waiting 12.33s (0 attempts) for RPC Merge

kvoli commented 2 weeks ago

Adding testing issue labels based on the above comment.