cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.18k stars 3.82k forks source link

roachtest: follower-reads/mixed-version/survival=region/locality=global/reads=strong failed #136099

Open cockroach-teamcity opened 2 days ago

cockroach-teamcity commented 2 days ago

roachtest.follower-reads/mixed-version/survival=region/locality=global/reads=strong failed with artifacts on master @ f717f6bd218121bb5e3376af658545f6bff30c22:

(follower_reads.go:757).verifySQLLatency: 9 latency values ([69.362926ms 70.636091ms 70.636091ms 70.636091ms 92.491472ms 92.491472ms 92.491472ms 92.491472ms 92.491472ms]) are above target latency 25ms, 4 permitted
(mixedversion.go:759).Run: panic (stack trace above): t.Fatal() was called
test artifacts and logs in: /artifacts/follower-reads/mixed-version/survival=region/locality=global/reads=strong/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #133520 roachtest: follower-reads/mixed-version/survival=region/locality=global/reads=strong failed [C-bug C-test-failure O-roachtest O-robot P-2 T-kv branch-release-24.3] - #132738 roachtest: follower-reads/mixed-version/survival=region/locality=global/reads=strong failed [C-test-failure O-roachtest O-robot T-testeng branch-release-23.2.13-rc]

/cc @cockroachdb/kv-triage

This test on roachdash | Improve this report!

Jira issue: CRDB-44889

andrewbaptist commented 1 day ago

Dupe of #133520. Still unclear why this is doing bad follower reads. I'll track down later this week.