roachtest: jepsen/bank/strobe-skews failed

[cockroach-teamcity]

roachtest.jepsen/bank/strobe-skews failed with artifacts on release-22.2.0 @ 22d02d78ecdd0b01f05272f54abafa097eec6720:

        (1) attached stack trace
          -- stack trace:
          | main.(*clusterImpl).RunE
          |     main/pkg/cmd/roachtest/cluster.go:2018
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runJepsen.func1
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/jepsen.go:171
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runJepsen.func3
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/jepsen.go:209
          | runtime.goexit
          |     GOROOT/src/runtime/asm_amd64.s:1594
        Wraps: (2) output in run_112641.851818446_n6_bash
        Wraps: (3) bash -e -c "\
          | cd /mnt/data1/jepsen/cockroachdb && set -eo pipefail && \
          |  ~/lein run test \
          |    --tarball file://${PWD}/cockroach.tgz \
          |    --username ${USER} \
          |    --ssh-private-key ~/.ssh/id_rsa \
          |    --os ubuntu \
          |    --time-limit 300 \
          |    --concurrency 30 \
          |    --recovery-time 25 \
          |    --test-count 1 \
          |    -n 10.142.0.53 -n 10.142.0.148 -n 10.142.0.123 -n 10.142.0.52 -n 10.142.0.122 \
          |    --test bank --nemesis strobe-skews \
          | > invoke.log 2>&1 \
          | " returned
          | stderr:
          |
          | stdout:
        Wraps: (4) SSH_PROBLEM
        Wraps: (5) Node 6. Command with error:
          | ``````
          | bash -e -c "\
          | cd /mnt/data1/jepsen/cockroachdb && set -eo pipefail && \
          |  ~/lein run test \
          |    --tarball file://${PWD}/cockroach.tgz \
          |    --username ${USER} \
          |    --ssh-private-key ~/.ssh/id_rsa \
          |    --os ubuntu \
          |    --time-limit 300 \
          |    --concurrency 30 \
          |    --recovery-time 25 \
          |    --test-count 1 \
          |    -n 10.142.0.53 -n 10.142.0.148 -n 10.142.0.123 -n 10.142.0.52 -n 10.142.0.122 \
          |    --test bank --nemesis strobe-skews \
          | > invoke.log 2>&1 \
          | "
          | ``````
        Wraps: (6) exit status 255
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) errors.SSH (5) *hintdetail.withDetail (6) *exec.ExitError

Parameters: ROACHTEST_cloud=gce , ROACHTEST_cpu=4 , ROACHTEST_encrypted=false , ROACHTEST_fs=ext4 , ROACHTEST_localSSD=true , ROACHTEST_ssd=0

Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

/cc @cockroachdb/test-eng

This test on roachdash | Improve this report!

Jira issue: CRDB-20989

[renatolabs]

jepsen tries to download dependencies using apt-get on the fly. In this run, it failed to install faketime.

ERROR [2022-10-29 11:26:59,394] main - jepsen.cli Oh jeez, I'm sorry, Jepsen broke. Here's why:
java.util.concurrent.ExecutionException: java.lang.RuntimeException: sudo -S -u root bash -x -c "cd /; env DEBIAN_FRONTEND=noninteractive apt-get install -y --force-yes ntpdate faketime unzip" returned non-zero exit status 100 on 10.142.0.53. STDOUT:
Reading package lists...
Building dependency tree...
Reading state information...

STDERR:
+ cd /
+ env DEBIAN_FRONTEND=noninteractive apt-get install -y --force-yes ntpdate faketime unzip
W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Unable to locate package faketime

[srosenberg]

same issue from last week: https://github.com/cockroachdb/cockroach/issues/90035#issuecomment-1280120192