cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.96k stars 3.79k forks source link

roachtest: multitenant-upgrade failed #130140

Open cockroach-teamcity opened 3 weeks ago

cockroach-teamcity commented 3 weeks ago

roachtest.multitenant-upgrade failed with artifacts on release-24.2.1-rc @ 95e6fb913c83be95110e0a250bcc75bbe863c8b1:

(sql_runner.go:257).Query: error executing 'SELECT version = crdb_internal.node_executable_version() FROM [SHOW CLUSTER SETTING version]': pq: operation "show cluster setting version" timed out after 2m0.001s (given timeout 2m0s): value differs between local setting (24.2-upgrading-step--01) and KV (24.1-upgrading-to-24.2-step-012)
(cluster.go:2434).Run: context canceled
(cluster.go:2434).Run: context canceled
(cluster.go:2434).Run: context canceled
(cluster.go:2434).Run: context canceled
test artifacts and logs in: /artifacts/multitenant-upgrade/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

/cc @cockroachdb/disaster-recovery

This test on roachdash | Improve this report!

Jira issue: CRDB-41889

msbutler commented 3 weeks ago

This should have been fixed by https://github.com/cockroachdb/cockroach/pull/129520 and didn't make the rc cut:

~/go/src/github.com/cockroachdb/master release-24.2 ⇣1⇡414 *1
❯ git log --oneline | grep "fix checkClusterSettingValuesAreEquivalent for final release"
c4a5721dfe2 sql: fix checkClusterSettingValuesAreEquivalent for final release fence versions

~/go/src/github.com/cockroachdb/master release-24.2 ⇣1⇡414 *1
❯ git switch release-24.2.1-rc
Switched to branch 'release-24.2.1-rc'
Your branch is up to date with 'origin/release-24.2.1-rc'.

~/go/src/github.com/cockroachdb/master release-24.2.1-rc *1
❯ git log --oneline | grep "fix checkClusterSettingValuesAreEquivalent for final release"
msbutler commented 3 weeks ago

cc @RaduBerinde, in case this patch was intended to backport to the rc branch. If not, ignore me.