cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.13k stars 3.81k forks source link

roachtest: c2c/shutdown/dest/coordinator failed #131143

Closed cockroach-teamcity closed 1 month ago

cockroach-teamcity commented 1 month ago

roachtest.c2c/shutdown/dest/coordinator failed with artifacts on master @ 83589fb87caa92fb42e83994f1691978f37e4cbb:

                      |   |     src/runtime/asm_amd64.s:1695
                      | Wraps: (3) _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/c2c/shutdown/dest/coordinator/cpu_arch=arm64/run_1/ssh/ssh_070704.146261884_n9_sudo-aptget-install-.log)
                      | Wraps: (4) TRANSIENT_ERROR(ssh_problem)
                      | Wraps: (5) exit status 255
                      | Error types: (1) *hintdetail.withDetail (2) *withstack.withStack (3) *errutil.withPrefix (4) errors.TransientError (5) *exec.ExitError
                    Wraps: (6) Node 9. Command with error:
                      | ``````
                      |
                      | sudo apt-get install -qqy apt-transport-https &&
                      | sudo apt-get install -qqy software-properties-common &&
                      | sudo apt-get install -y adduser libfontconfig1 &&
                      | echo "Downloading https://dl.grafana.com/enterprise/release/grafana-enterprise_9.2.3_arm64.deb" &&
                      | curl https://dl.grafana.com/enterprise/release/grafana-enterprise_9.2.3_arm64.deb -sS -o grafana-enterprise_9.2.3_arm64.deb &&
                      | sudo dpkg -i grafana-enterprise_9.2.3_arm64.deb &&
                      | sudo mkdir -p /var/lib/grafana/dashboards
                      | ``````
                      | <truncated> ... sly unselected package grafana-enterprise.
                      | (Reading database ... 67950 files and directories currently installed.)
                      | Preparing to unpack grafana-enterprise_9.2.3_arm64.deb ...
                      | Unpacking grafana-enterprise (9.2.3) ...
                      |
                      | stderr:debconf: unable to initialize frontend: Dialog
                      | debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
                      | debconf: falling back to frontend: Readline
                      | debconf: unable to initialize frontend: Readline
                      | debconf: (This frontend requires a controlling tty.)
                      | debconf: falling back to frontend: Teletype
                      | dpkg-preconfigure: unable to re-open stdin: 
                      | debconf: unable to initialize frontend: Dialog
                      | debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
                      | debconf: falling back to frontend: Readline
                      | debconf: unable to initialize frontend: Readline
                      | debconf: (This frontend requires a controlling tty.)
                      | debconf: falling back to frontend: Teletype
                      | dpkg-preconfigure: unable to re-open stdin:
                    Wraps: (7) attached stack trace
                      -- stack trace:
                      | github.com/cockroachdb/cockroach/pkg/roachprod/install.(*remoteSession).errWithDebug
                      |     github.com/cockroachdb/cockroach/pkg/roachprod/install/session.go:143
                      | github.com/cockroachdb/cockroach/pkg/roachprod/install.(*remoteSession).Run.func1
                      |     github.com/cockroachdb/cockroach/pkg/roachprod/install/session.go:173
                      | runtime.goexit
                      |     src/runtime/asm_amd64.s:1695
                    Wraps: (8) _potential_ SSH flake (``ssh -vvv`` log retained in /artifacts/c2c/shutdown/dest/coordinator/cpu_arch=arm64/run_1/ssh/ssh_070243.942924485_n9_sudo-aptget-install-.log)
                    Wraps: (9) TRANSIENT_ERROR(ssh_problem)
                    Wraps: (10) exit status 255
                    Error types: (1) *markers.withMark (2) *withstack.withStack (3) *errutil.withPrefix (4) *secondary.withSecondaryError (5) *secondary.withSecondaryError (6) *hintdetail.withDetail (7) *withstack.withStack (8) *errutil.withPrefix (9) errors.TransientError (10) *exec.ExitError
    Test:           c2c/shutdown/dest/coordinator
(require.go:1357).NoError: FailNow called
test artifacts and logs in: /artifacts/c2c/shutdown/dest/coordinator/cpu_arch=arm64/run_1

Parameters:

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #128742 roachtest: c2c/shutdown/dest/coordinator failed [C-test-failure O-roachtest O-robot P-3 T-disaster-recovery branch-release-24.1.3-rc release-blocker]

/cc @cockroachdb/disaster-recovery

This test on roachdash | Improve this report!

Jira issue: CRDB-42402

msbutler commented 1 month ago

another ssh flake

blathers-crl[bot] commented 1 month ago

cc @cockroachdb/test-eng

blathers-crl[bot] commented 1 month ago

cc @cockroachdb/test-eng

renatolabs commented 1 month ago

Instance of #131095 (which is fixed on master as of yesterday).