cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.1k stars 3.81k forks source link

roachtest: copy/bank/rows=100000,nodes=5,txn=true failed #134402

Open cockroach-teamcity opened 22 hours ago

cockroach-teamcity commented 22 hours ago

roachtest.copy/bank/rows=100000,nodes=5,txn=true failed with artifacts on release-24.3 @ cb680f5c1811b890ecde369ec5e7d18e05ef1b37:

(cluster.go:2451).Run: full command output in run_115148.958423879_n1_cockroach-workload-f.log: COMMAND_PROBLEM: exit status 1
(monitor.go:149).Wait: monitor failure: monitor user task failed: t.Fatal() was called
test artifacts and logs in: /artifacts/copy/bank/rows=100000_nodes=5_txn=true/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #134387 roachtest: copy/bank/rows=100000,nodes=5,txn=true failed [C-test-failure O-roachtest O-robot T-kv branch-release-24.2.5-rc release-blocker] - #134385 roachtest: copy/bank/rows=100000,nodes=5,txn=true failed [C-test-failure O-roachtest O-robot T-kv branch-release-24.1.7-rc release-blocker] - #134378 roachtest: copy/bank/rows=100000,nodes=5,txn=true failed [C-test-failure O-roachtest O-robot T-kv branch-release-23.1 release-blocker] - #134375 roachtest: copy/bank/rows=100000,nodes=5,txn=true failed [C-test-failure O-roachtest O-robot T-kv branch-release-23.1.29-rc release-blocker]

/cc @cockroachdb/kv-triage

This test on roachdash | Improve this report!

Jira issue: CRDB-44059

arulajmani commented 19 hours ago
  |   | Error: restoring fixture: restore: gs://cockroach-fixtures-us-east1/workload/bank/version=1.0.0,payload-bytes=100,ranges=10,rows=100000,seed=1/bank?AUTH=implicit: pq: could not find BACKUP manifest file in any of the known locations: BACKUP_METADATA, BACKUP_MANIFEST, BACKUP: open workload/bank/version=1.0.0,payload-bytes=100,ranges=10,rows=100000,seed=1/bank/BACKUP: storage: object doesn't exist: gcs object "workload/bank/version=1.0.0,payload-bytes=100,ranges=10,rows=100000,seed=1/bank/BACKUP" does not exist: external_storage: file doesn't exist

Seems like something like this was fixed in https://github.com/cockroachdb/cockroach/pull/134215 by @kev-cao -- does this need to be backported to 24.3 as well? Either way, I'll move this to DR.

blathers-crl[bot] commented 19 hours ago

cc @cockroachdb/disaster-recovery