cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.16k stars 3.82k forks source link

roachtest: admission-control/disk-bandwidth-limiter failed #135792

Closed cockroach-teamcity closed 1 day ago

cockroach-teamcity commented 2 days ago

roachtest.admission-control/disk-bandwidth-limiter failed with artifacts on master @ 8eeb7f2ae3b2cede564b46ca47e2353fd147c061:

(cluster.go:2456).Run: full command output in run_064410.644421215_n2_cockroach-workload-r.log: COMMAND_PROBLEM: exit status 1
(cluster.go:2456).Run: context canceled
(admission_control_disk_bandwidth_overload.go:142).func1: failed to set kvadmission.store.provisioned_bandwidth: context canceled
test artifacts and logs in: /artifacts/admission-control/disk-bandwidth-limiter/cpu_arch=arm64/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #135396 roachtest: admission-control/disk-bandwidth-limiter failed [A-storage C-test-failure O-roachtest O-robot T-storage branch-release-24.3] - #134932 roachtest: admission-control/disk-bandwidth-limiter failed [A-storage C-test-failure O-roachtest O-robot T-storage branch-release-24.3.0-rc]

/cc @cockroachdb/storage

This test on roachdash | Improve this report!

Jira issue: CRDB-44711

cockroach-teamcity commented 2 days ago

roachtest.admission-control/disk-bandwidth-limiter failed with artifacts on master @ 8eeb7f2ae3b2cede564b46ca47e2353fd147c061:

(cluster.go:2456).Run: full command output in run_070024.612658639_n2_cockroach-workload-r.log: COMMAND_PROBLEM: exit status 1
(cluster.go:2456).Run: context canceled
(admission_control_disk_bandwidth_overload.go:142).func1: failed to set kvadmission.store.provisioned_bandwidth: context canceled
test artifacts and logs in: /artifacts/admission-control/disk-bandwidth-limiter/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

Grafana is not yet available for aws clusters

Same failure on other branches

- #135396 roachtest: admission-control/disk-bandwidth-limiter failed [A-storage C-test-failure O-roachtest O-robot T-storage branch-release-24.3] - #134932 roachtest: admission-control/disk-bandwidth-limiter failed [A-storage C-test-failure O-roachtest O-robot T-storage branch-release-24.3.0-rc]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 day ago

roachtest.admission-control/disk-bandwidth-limiter failed with artifacts on master @ eb2d2e19eb29d2747d9e267bd0612a69d066adad:

(cluster.go:2456).Run: full command output in run_064815.441757085_n2_cockroach-workload-r.log: COMMAND_PROBLEM: exit status 1
(cluster.go:2456).Run: context canceled
(admission_control_disk_bandwidth_overload.go:142).func1: failed to set kvadmission.store.provisioned_bandwidth: context canceled
test artifacts and logs in: /artifacts/admission-control/disk-bandwidth-limiter/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #135396 roachtest: admission-control/disk-bandwidth-limiter failed [A-storage C-test-failure O-roachtest O-robot T-storage branch-release-24.3] - #134932 roachtest: admission-control/disk-bandwidth-limiter failed [A-storage C-test-failure O-roachtest O-robot T-storage branch-release-24.3.0-rc]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 day ago

roachtest.admission-control/disk-bandwidth-limiter failed with artifacts on master @ eb2d2e19eb29d2747d9e267bd0612a69d066adad:

(cluster.go:2456).Run: full command output in run_065910.529360660_n2_cockroach-workload-r.log: COMMAND_PROBLEM: exit status 1
(cluster.go:2456).Run: context canceled
(admission_control_disk_bandwidth_overload.go:142).func1: failed to set kvadmission.store.provisioned_bandwidth: context canceled
test artifacts and logs in: /artifacts/admission-control/disk-bandwidth-limiter/cpu_arch=arm64/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

Grafana is not yet available for aws clusters

Same failure on other branches

- #135396 roachtest: admission-control/disk-bandwidth-limiter failed [A-storage C-test-failure O-roachtest O-robot T-storage branch-release-24.3] - #134932 roachtest: admission-control/disk-bandwidth-limiter failed [A-storage C-test-failure O-roachtest O-robot T-storage branch-release-24.3.0-rc]

This test on roachdash | Improve this report!

aadityasondhi commented 1 day ago
max-block-bytes=1024 --txn-qos='regular' --tolerate-errors  --db='regular_kv'  --duration=1h30m0s  {pgurl:1}
I241121 06:59:11.227354 1 workload/cli/run.go:665  [-] 1  random seed: 1360592049259520826
Error: error creating metrics exporter: file path must end with .json
run_065910.529360660_n2_cockroach-workload-r: 2024/11/21 06:59:11 cluster.go:2493: > result: COMMAND_PROBLEM: exit status 1

Thought this was fixed. @cockroachdb/test-eng Does anyone have a clue? The is from the failure 7 hours ago.