cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.13k stars 3.81k forks source link

roachtest: admission-control/disk-bandwidth-limiter failed #134932

Open cockroach-teamcity opened 2 days ago

cockroach-teamcity commented 2 days ago

roachtest.admission-control/disk-bandwidth-limiter failed with artifacts on release-24.3.0-rc @ 6d0a627cbefef712d05815b90b738e01805a618e:

(admission_control_disk_bandwidth_overload.go:185).3: write + read bandwidth 81.800331 (66.691227 + 15.109104) exceeded threshold of 78.750000
(cluster.go:2451).Run: context canceled
(cluster.go:2451).Run: context canceled
(monitor.go:149).Wait: monitor failure: monitor user task failed: t.Fatal() was called
test artifacts and logs in: /artifacts/admission-control/disk-bandwidth-limiter/cpu_arch=arm64/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

Grafana is not yet available for aws clusters

Same failure on other branches

- #132977 roachtest: admission-control/disk-bandwidth-limiter failed [A-admission-control A-storage C-test-failure O-roachtest O-robot P-1 T-storage branch-release-24.3] - #131484 roachtest: admission-control/disk-bandwidth-limiter failed [A-storage C-test-failure O-roachtest O-robot P-1 T-storage branch-master]

/cc @cockroachdb/storage

This test on roachdash | Improve this report!

Jira issue: CRDB-44298

aadityasondhi commented 2 days ago

Fix is already merged to both master and 24.3.

https://github.com/cockroachdb/cockroach/pull/134430 https://github.com/cockroachdb/cockroach/pull/134973

cockroach-teamcity commented 1 day ago

roachtest.admission-control/disk-bandwidth-limiter failed with artifacts on release-24.3.0-rc @ 01e6bc2badb8c14b128a52481f59cbbebf879dd8:

(admission_control_disk_bandwidth_overload.go:185).3: write + read bandwidth 80.512578 (66.277969 + 14.234609) exceeded threshold of 78.750000
(cluster.go:2451).Run: context canceled
(cluster.go:2451).Run: context canceled
(monitor.go:149).Wait: monitor failure: monitor user task failed: t.Fatal() was called
test artifacts and logs in: /artifacts/admission-control/disk-bandwidth-limiter/cpu_arch=arm64/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

Grafana is not yet available for aws clusters

Same failure on other branches

- #135056 roachtest: admission-control/disk-bandwidth-limiter failed [A-storage C-test-failure O-roachtest O-robot T-storage branch-master release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 20 hours ago

roachtest.admission-control/disk-bandwidth-limiter failed with artifacts on release-24.3.0-rc @ 516de012609eb9463b2df0fdc78958ca0549074e:

(admission_control_disk_bandwidth_overload.go:185).3: write + read bandwidth 79.092109 (67.064609 + 12.027500) exceeded threshold of 78.750000
(cluster.go:2451).Run: context canceled
(cluster.go:2451).Run: context canceled
(monitor.go:149).Wait: monitor failure: monitor user task failed: t.Fatal() was called
test artifacts and logs in: /artifacts/admission-control/disk-bandwidth-limiter/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #135056 roachtest: admission-control/disk-bandwidth-limiter failed [A-storage C-test-failure O-roachtest O-robot T-storage branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 20 hours ago

roachtest.admission-control/disk-bandwidth-limiter failed with artifacts on release-24.3.0-rc @ 516de012609eb9463b2df0fdc78958ca0549074e:

(admission_control_disk_bandwidth_overload.go:185).3: write + read bandwidth 86.745625 (51.886953 + 34.858672) exceeded threshold of 78.750000
(cluster.go:2451).Run: context canceled
(cluster.go:2451).Run: context canceled
(monitor.go:149).Wait: monitor failure: monitor user task failed: t.Fatal() was called
test artifacts and logs in: /artifacts/admission-control/disk-bandwidth-limiter/cpu_arch=arm64/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

Grafana is not yet available for aws clusters

Same failure on other branches

- #135056 roachtest: admission-control/disk-bandwidth-limiter failed [A-storage C-test-failure O-roachtest O-robot T-storage branch-master]

This test on roachdash | Improve this report!