search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.9k stars 3.78k forks source link

roachtest: vm_preemption failed #130462

Closed cockroach-teamcity closed 5 days ago

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test allocbench/nodes=7/cpu=8/kv/r=0/ops=skew failed: VMs preempted during the test run: teamcity-16830221-1726033678-08-n8cpu8-0005 (us-east1-d)

**Other Failures:**
(cluster.go:2305).Start: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/allocbench/nodes=7/cpu=8/kv/r=0/ops=skew/run_1/ssh/ssh_055623.536526844_n5_upload-start-script.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/allocbench/nodes=7/cpu=8/kv/r=0/ops=skew/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

/cc @cockroachdb/test-eng

This test on roachdash | Improve this report!

Jira issue: CRDB-42060

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test alterpk-tpcc-250 failed: VMs preempted during the test run: teamcity-16830221-1726033678-10-n4cpu32-0002 (us-east1-d)

**Other Failures:**
(cluster.go:2305).Start: one or more parallel execution failure(s): error persisted after 3 attempts: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/alterpk-tpcc-250/run_1/ssh/ssh_055713.444044367_n2_dist-local-certs.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/alterpk-tpcc-250/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test backup/KMS/gce/n3cpu4 failed: VMs preempted during the test run: teamcity-16830221-1726033678-16-n3cpu4-0003 (us-east1-d)

**Other Failures:**
(monitor.go:154).Wait: monitor failure: pq: query execution canceled
EOF [owner=test-eng]
test artifacts and logs in: /artifacts/backup/KMS/gce/n3cpu4/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test cdc/initial-scan-only/parquet failed: VMs preempted during the test run: teamcity-16830221-1726033678-24-n4cpu16-0002 (us-east1-d), teamcity-16830221-1726033678-24-n4cpu16-0001 (us-east1-d), teamcity-16830221-1726033678-24-n4cpu16-0004 (us-east1-d)

**Other Failures:**
(monitor.go:154).Wait: monitor failure: read tcp 172.17.0.3:47388 -> 35.196.247.123:26257: read: connection reset by peer
test artifacts and logs in: /artifacts/cdc/initial-scan-only/parquet/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test jobs/stress failed: VMs preempted during the test run: teamcity-16830221-1726033678-04-n4cpu4-0004 (us-east1-c)

**Other Failures:**
(monitor.go:154).Wait: monitor failure: context canceled
EOF [owner=test-eng]
test artifacts and logs in: /artifacts/jobs/stress/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test cdc/scan/catchup-cold/nodes=5/cpu=16/rows=1G/ranges=100/protocol=mux/format=json/sink=null failed: VMs preempted during the test run: teamcity-16830221-1726033678-26-n6cpu16-0005 (us-east1-d)

**Other Failures:**
(monitor.go:154).Wait: monitor failure: context canceled
failed to list cockroach processes: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/cdc/scan/catchup-cold/nodes=5/cpu=16/rows=1G/ranges=100/protocol=mux/format=json/sink=null/run_1/ssh/ssh_064212.130146405_n5_list-processes.log): TRANSIENT_ERROR(ssh_problem): exit status 255 [owner=test-eng]
test artifacts and logs in: /artifacts/cdc/scan/catchup-cold/nodes=5/cpu=16/rows=1G/ranges=100/protocol=mux/format=json/sink=null/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test cdc/scan/catchup/nodes=5/cpu=16/rows=1G/ranges=100k/protocol=mux/format=json/sink=null failed: VMs preempted during the test run: teamcity-16830221-1726033678-28-n6cpu16-0002 (us-east1-c)

**Other Failures:**
(monitor.go:154).Wait: monitor failure: context canceled
failed to list cockroach processes: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/cdc/scan/catchup/nodes=5/cpu=16/rows=1G/ranges=100k/protocol=mux/format=json/sink=null/run_1/ssh/ssh_073328.317159142_n2_list-processes.log): TRANSIENT_ERROR(ssh_problem): exit status 255 [owner=test-eng]
test artifacts and logs in: /artifacts/cdc/scan/catchup/nodes=5/cpu=16/rows=1G/ranges=100k/protocol=mux/format=json/sink=null/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test tpccbench/nodes=3/cpu=4/backups=true failed: VMs preempted during the test run: teamcity-16830221-1726033678-03-n4cpu4-0003 (us-east1-b)

**Other Failures:**
(cluster.go:2355).Stop: cluster.StopE: one or more parallel execution failure(s): _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/tpccbench/nodes=3/cpu=4/backups=true/run_1/ssh/ssh_075335.522048362_n3_kill.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/tpccbench/nodes=3/cpu=4/backups=true/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test tpch_concurrency/no_streamer failed: VMs preempted during the test run: teamcity-16830221-1726033678-09-n4cpu4-0001 (us-east1-b)

**Other Failures:**
(cluster.go:2355).Stop: cluster.StopE: one or more parallel execution failure(s): _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/tpch_concurrency/no_streamer/run_1/ssh/ssh_075324.944600383_n1_kill.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/tpch_concurrency/no_streamer/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test drop/tpcc/w=100,nodes=9 failed: VMs preempted during the test run: teamcity-16830221-1726033678-49-n9cpu4-0001 (us-central1-b), teamcity-16830221-1726033678-49-n9cpu4-0007 (us-central1-b)

**Other Failures:**
(cluster.go:2418).Run: context canceled
(monitor.go:154).Wait: monitor failure: monitor user task failed: t.Fatal() was called
EOF [owner=test-eng]
test artifacts and logs in: /artifacts/drop/tpcc/w=100_nodes=9/cpu_arch=arm64/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test failover/non-system/blackhole-recv failed: VMs preempted during the test run: teamcity-16830221-1726033678-53-n7cpu2-0001 (us-east1-c)

**Other Failures:**
(failover.go:1782).sleepFor: sleep failed: context canceled
(monitor.go:154).Wait: monitor failure: monitor user task failed: t.Fatal() was called
EOF [owner=test-eng]
(cluster.go:2418).Run: context canceled
test artifacts and logs in: /artifacts/failover/non-system/blackhole-recv/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test kv95limited-spans/enc=false/nodes=1/seq/no-load-splitting failed: VMs preempted during the test run: teamcity-16830221-1726033678-06-n2cpu8-0001 (us-east1-b), teamcity-16830221-1726033678-06-n2cpu8-0002 (us-east1-b)

**Other Failures:**
(cluster.go:2418).Run: context canceled
(monitor.go:154).Wait: monitor failure: monitor user task failed: t.Fatal() was called
EOF [owner=test-eng]
test artifacts and logs in: /artifacts/kv95limited-spans/enc=false/nodes=1/seq/no-load-splitting/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test schemachange/during/tpcc failed: VMs preempted during the test run: teamcity-16830221-1726033678-44-n5cpu16-0003 (us-east1-b), teamcity-16830221-1726033678-44-n5cpu16-0002 (us-east1-b)

**Other Failures:**
(monitor.go:154).Wait: monitor failure: context canceled
EOF [owner=test-eng]
test artifacts and logs in: /artifacts/schemachange/during/tpcc/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test failover/non-system/blackhole/lease=expiration failed: VMs preempted during the test run: teamcity-16830221-1726033678-54-n7cpu2-0003 (us-east1-b), teamcity-16830221-1726033678-54-n7cpu2-0002 (us-east1-b)

**Other Failures:**
(failover.go:1782).sleepFor: sleep failed: context canceled
(monitor.go:154).Wait: monitor failure: monitor user task failed: t.Fatal() was called
EOF [owner=test-eng]
(cluster.go:2418).Run: context canceled
test artifacts and logs in: /artifacts/failover/non-system/blackhole/lease=expiration/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test jepsen/bank-multitable/parts-start-kill-2 failed: VMs preempted during the test run: teamcity-16830221-1726033678-79-n6cpu4-0006 (us-central1-a)

**Other Failures:**
(jepsen.go:332).func2: full command output in run_092943.443625149_n6_tail-n-100-mntdata1j.log: error persisted after 3 attempts: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/jepsen/bank-multitable/parts-start-kill-2/cpu_arch=arm64/run_1/ssh/ssh_092943.492037550_n6_tail-n-100-mntdata1j.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/jepsen/bank-multitable/parts-start-kill-2/cpu_arch=arm64/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test kv0/enc=false/nodes=3/cpu=32/size=64kb failed: VMs preempted during the test run: teamcity-16830221-1726033678-102-n4cpu32-0002 (us-east1-d)

**Other Failures:**
(cluster.go:2418).Run: context canceled
(monitor.go:154).Wait: monitor failure: monitor user task failed: t.Fatal() was called
EOF [owner=test-eng]
test artifacts and logs in: /artifacts/kv0/enc=false/nodes=3/cpu=32/size=64kb/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test ycsb/C/nodes=3/cpu=32/isolation-level=read-committed failed: VMs preempted during the test run: teamcity-16830221-1726033678-153-n4cpu32-0004 (us-east1-b)

**Other Failures:**
(cluster.go:2418).Run: full command output in run_115222.060135737_n4_cockroach-workload-r.log: error persisted after 3 attempts: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/ycsb/C/nodes=3/cpu=32/isolation-level=read-committed/run_1/ssh/ssh_115222.077792313_n4_cockroach-workload-r.log): TRANSIENT_ERROR(ssh_problem): exit status 255
(monitor.go:154).Wait: monitor failure: monitor user task failed: t.Fatal() was called
test artifacts and logs in: /artifacts/ycsb/C/nodes=3/cpu=32/isolation-level=read-committed/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test ycsb/F/nodes=3/cpu=32/isolation-level=read-committed failed: VMs preempted during the test run: teamcity-16830221-1726033678-157-n4cpu32-0002 (us-east1-b)

**Other Failures:**
(cluster.go:2418).Run: context canceled
(monitor.go:154).Wait: monitor failure: monitor user task failed: t.Fatal() was called
EOF [owner=test-eng]
test artifacts and logs in: /artifacts/ycsb/F/nodes=3/cpu=32/isolation-level=read-committed/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test tpccbench/nodes=9/cpu=4/multi-region failed: VMs preempted during the test run: teamcity-16830221-1726033678-144-n12cpu4hm-geo-0004 (us-east1-b), teamcity-16830221-1726033678-144-n12cpu4hm-geo-0002 (us-east1-b), teamcity-16830221-1726033678-144-n12cpu4hm-geo-0003 (us-east1-b)

**Other Failures:**
(cluster.go:2305).Start: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/tpccbench/nodes=9/cpu=4/multi-region/run_1/ssh/ssh_143253.396817610_n2_upload-start-script.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/tpccbench/nodes=9/cpu=4/multi-region/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test import/tpcc/warehouses=1000/nodes=4 failed: VMs preempted during the test run: teamcity-16847050-1726120084-03-n4cpu4-0004 (us-east1-c)

**Other Failures:**
(cluster.go:2418).Run: context canceled
(monitor.go:154).Wait: monitor failure: context canceled
EOF [owner=test-eng]
test artifacts and logs in: /artifacts/import/tpcc/warehouses=1000/nodes=4/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test allocbench/nodes=7/cpu=8/kv/r=50/ops=skew failed: VMs preempted during the test run: teamcity-16847050-1726120084-12-n8cpu8-0004 (us-east1-c), teamcity-16847050-1726120084-12-n8cpu8-0001 (us-east1-c)

**Other Failures:**
(assertions.go:363).Fail: 
    Error Trace:    github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/allocation_bench.go:389
                                github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/allocation_bench.go:339
                                github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/allocation_bench.go:268
                                main/pkg/cmd/roachtest/test_runner.go:1255
                                src/runtime/asm_amd64.s:1695
    Error:          Received unexpected error:
                    monitor failure: context canceled
                    (1) attached stack trace
                      -- stack trace:
                      | main.(*monitorImpl).WaitE
                      |     main/pkg/cmd/roachtest/monitor.go:142
                      | [...repeated from below...]
                    Wraps: (2) monitor failure
                    Wraps: (3) attached stack trace
                      -- stack trace:
                      | main.(*monitorImpl).wait
                      |     main/pkg/cmd/roachtest/monitor.go:237
                      | main.(*monitorImpl).WaitE
                      |     main/pkg/cmd/roachtest/monitor.go:142
                      | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runAllocationBenchSample
                      |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/allocation_bench.go:388
                      | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runAllocationBench
                      |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/allocation_bench.go:339
                      | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerAllocationBenchSpec.func1
                      |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/allocation_bench.go:268
                      | main.(*testRunner).runTest.func2
                      |     main/pkg/cmd/roachtest/test_runner.go:1255
                      | runtime.goexit
                      |     src/runtime/asm_amd64.s:1695
                    Wraps: (4) context canceled
                      | EOF [owner=test-eng]
                        └─ Wraps: (5) EOF [owner=test-eng]
                        └─ Wraps: (6) context canceled
                    Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *join.joinError (5) registry.ErrorWithOwnership (6) *errors.errorString
    Test:           allocbench/nodes=7/cpu=8/kv/r=50/ops=skew
(require.go:1357).NoError: FailNow called
test artifacts and logs in: /artifacts/allocbench/nodes=7/cpu=8/kv/r=50/ops=skew/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test alterpk-tpcc-250 failed: VMs preempted during the test run: teamcity-16847050-1726120084-14-n4cpu32-0003 (us-east1-c), teamcity-16847050-1726120084-14-n4cpu32-0002 (us-east1-c)

**Other Failures:**
(cluster.go:2418).Run: context canceled
(monitor.go:154).Wait: monitor failure: monitor user task failed: t.Fatal() was called
EOF [owner=test-eng]
test artifacts and logs in: /artifacts/alterpk-tpcc-250/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test buffered_logging failed: VMs preempted during the test run: teamcity-16847050-1726120084-13-n4cpu4-0004 (us-east1-c), teamcity-16847050-1726120084-13-n4cpu4-0002 (us-east1-c), teamcity-16847050-1726120084-13-n4cpu4-0001 (us-east1-c)

**Other Failures:**
(monitor.go:154).Wait: monitor failure: context canceled
EOF [owner=test-eng]
test artifacts and logs in: /artifacts/buffered_logging/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test sqlsmith/setup=seed/setting=no-ddl failed: VMs preempted during the test run: teamcity-16847050-1726120084-07-n4cpu4-0003 (us-east1-c), teamcity-16847050-1726120084-07-n4cpu4-0001 (us-east1-c)

**Other Failures:**
(sqlsmith.go:324).func3: ping node 3: driver: bad connection
HINT: node likely crashed, check logs in artifacts > logs/3.unredacted
previous sql:
EXPLAIN INSERT
INTO
    defaultdb.public.seed AS tab_2801 (_int2, _int8, _float4, _uuid, _enum)
VALUES
    (
        (-867):::INT8,
        (-6041676989194099092):::INT8,
        (-0.47899356484413147):::FLOAT8,
        'b4aa7b99-329a-4df2-b25c-ec77d2bc9f0a':::UUID,
        'morning':::greeting
    );ping node 3: driver: bad connection
HINT: node likely crashed, check logs in artifacts > logs/3.unredacted
test artifacts and logs in: /artifacts/sqlsmith/setup=seed/setting=no-ddl/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test hibernate failed: VMs preempted during the test run: teamcity-16847050-1726120084-08-n1cpu4-0001 (us-east1-c)

**Other Failures:**
(hibernate.go:173).func1: all attempts failed for building hibernate (without tests): full command output in run_063928.406741328_n1_cd-mntdata1hibernate.log: error persisted after 3 attempts: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/hibernate/run_1/ssh/ssh_063928.450012100_n1_cd-mntdata1hibernate.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/hibernate/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test tpccbench/nodes=3/cpu=4 failed: VMs preempted during the test run: teamcity-16847050-1726120084-01-n4cpu4-0001 (us-east1-b)

**Other Failures:**
(monitor.go:154).Wait: monitor failure: context canceled
EOF [owner=test-eng]
test artifacts and logs in: /artifacts/tpccbench/nodes=3/cpu=4/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test cdc/workload/kv0/nodes=5/cpu=16/ranges=100k/server=scheduler/protocol=mux/format=json/sink=null failed: VMs preempted during the test run: teamcity-16847050-1726120084-33-n7cpu16-0002 (us-east1-b)

**Other Failures:**
(monitor.go:154).Wait: monitor failure: context canceled
failed to list cockroach processes: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/cdc/workload/kv0/nodes=5/cpu=16/ranges=100k/server=scheduler/protocol=mux/format=json/sink=null/run_1/ssh/ssh_073550.352149634_n2_list-processes.log): TRANSIENT_ERROR(ssh_problem): exit status 255 [owner=test-eng]
test artifacts and logs in: /artifacts/cdc/workload/kv0/nodes=5/cpu=16/ranges=100k/server=scheduler/protocol=mux/format=json/sink=null/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test cdc/scan/catchup/nodes=5/cpu=16/rows=1G/ranges=100k/protocol=mux/format=json/sink=null failed: VMs preempted during the test run: teamcity-16847050-1726120084-26-n6cpu16-0001 (us-east1-c), teamcity-16847050-1726120084-26-n6cpu16-0005 (us-east1-c), teamcity-16847050-1726120084-26-n6cpu16-0006 (us-east1-c)

**Other Failures:**
(monitor.go:154).Wait: monitor failure: context canceled
EOF [owner=test-eng]
test artifacts and logs in: /artifacts/cdc/scan/catchup/nodes=5/cpu=16/rows=1G/ranges=100k/protocol=mux/format=json/sink=null/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test cdc/workload/kv0/nodes=5/cpu=16/ranges=100k/server=processor/protocol=mux/format=json/sink=null failed: VMs preempted during the test run: teamcity-16847050-1726120084-34-n7cpu16-0005 (us-east1-c)

**Other Failures:**
(monitor.go:154).Wait: monitor failure: context canceled
EOF [owner=test-eng]
test artifacts and logs in: /artifacts/cdc/workload/kv0/nodes=5/cpu=16/ranges=100k/server=processor/protocol=mux/format=json/sink=null/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test clearrange/checks=true failed: VMs preempted during the test run: teamcity-16847050-1726120084-47-n10cpu16-0008 (us-east1-d)

**Other Failures:**
(cluster.go:2418).Run: context canceled
(monitor.go:154).Wait: monitor failure: monitor user task failed: t.Fatal() was called
EOF [owner=test-eng]
test artifacts and logs in: /artifacts/clearrange/checks=true/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test jepsen/bank-multitable/start-stop-2 failed: VMs preempted during the test run: teamcity-16847050-1726120084-77-n6cpu4-0001 (us-central1-a)

**Other Failures:**
(cluster.go:1878).Put: cluster.PutE: error persisted after 3 attempts: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/jepsen/bank-multitable/start-stop-2/cpu_arch=arm64/run_1/ssh/ssh_091651.367980306_n1_if-hostname-teamcity.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/jepsen/bank-multitable/start-stop-2/cpu_arch=arm64/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test jepsen/g2/subcritical-skews-start-kill-2 failed: VMs preempted during the test run: teamcity-16847050-1726120084-78-n6cpu4-0005 (us-central1-b)

**Other Failures:**
(cluster.go:2418).Run: full command output in run_093418.185143886_n1-6_mkdir-p-logs.log: error persisted after 3 attempts: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/jepsen/g2/subcritical-skews-start-kill-2/cpu_arch=arm64/run_1/ssh/ssh_093418.232809879_n5_mkdir-p-logs.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/jepsen/g2/subcritical-skews-start-kill-2/cpu_arch=arm64/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test kv0/enc=false/nodes=3/mt-shared-process failed: VMs preempted during the test run: teamcity-16847050-1726120084-100-n4cpu8-0004 (us-east1-c)

**Other Failures:**
(cluster.go:2418).Run: full command output in run_101019.916550366_n4_cockroach-workload-r.log: error persisted after 3 attempts: _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/kv0/enc=false/nodes=3/mt-shared-process/run_1/ssh/ssh_101019.942517560_n4_cockroach-workload-r.log): TRANSIENT_ERROR(ssh_problem): exit status 255
(monitor.go:154).Wait: monitor failure: monitor user task failed: t.Fatal() was called
test artifacts and logs in: /artifacts/kv0/enc=false/nodes=3/mt-shared-process/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test kv95/enc=false/nodes=3/batch=16 failed: VMs preempted during the test run: teamcity-16847050-1726120084-106-n4cpu8-0001 (us-east1-d), teamcity-16847050-1726120084-106-n4cpu8-0002 (us-east1-d)

**Other Failures:**
(cluster.go:2418).Run: context canceled
(monitor.go:154).Wait: monitor failure: monitor user task failed: t.Fatal() was called
EOF [owner=test-eng]
test artifacts and logs in: /artifacts/kv95/enc=false/nodes=3/batch=16/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test tpch_concurrency/no_streamer failed: VMs preempted during the test run: teamcity-16847050-1726120084-149-n4cpu4-0001 (us-east1-c), teamcity-16847050-1726120084-149-n4cpu4-0004 (us-east1-c)

**Other Failures:**
(cluster.go:2355).Stop: cluster.StopE: one or more parallel execution failure(s): _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/tpch_concurrency/no_streamer/run_1/ssh/ssh_121245.625331598_n1_kill.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/tpch_concurrency/no_streamer/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test tpccbench/nodes=9/cpu=4/multi-region failed: VMs preempted during the test run: teamcity-16847050-1726120084-147-n12cpu4hm-geo-0003 (us-east1-b)

**Other Failures:**
(monitor.go:154).Wait: monitor failure: context canceled
EOF [owner=test-eng]
test artifacts and logs in: /artifacts/tpccbench/nodes=9/cpu=4/multi-region/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

roachtest.vm_preemption failed with artifacts on release-24.2.2-rc @ 11f594a0ef80362f76260b6720d48e633292d626:

test tpch_concurrency failed: VMs preempted during the test run: teamcity-16847050-1726120084-148-n4cpu4-0003 (us-east1-c), teamcity-16847050-1726120084-148-n4cpu4-0002 (us-east1-c)

**Other Failures:**
(cluster.go:2355).Stop: cluster.StopE: one or more parallel execution failure(s): _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/tpch_concurrency/run_1/ssh/ssh_125632.359350017_n2_kill.log): TRANSIENT_ERROR(ssh_problem): exit status 255
test artifacts and logs in: /artifacts/tpch_concurrency/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #130467 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.5-rc] - #129304 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2.1-rc] - #127323 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.3-rc] - #126985 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.2] - #125727 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.2-rc] - #124898 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1.1-rc] - #123741 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #121348 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-release-24.1] - #118528 roachtest: vm_preemption failed [O-roachtest O-robot T-testeng X-infra-flake branch-master]

This test on roachdash | Improve this report!

renatolabs commented 5 days ago

Branch no longer exists, closing.