cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.87k stars 3.77k forks source link

roachtest: autoupgrade failed #92801

Closed cockroach-teamcity closed 1 year ago

cockroach-teamcity commented 1 year ago

roachtest.autoupgrade failed with artifacts on release-22.1 @ f9730bda77ce8e3ecfa42302be58d81dcd04cd21:

The test failed on branch=release-22.1, cloud=gce:
test artifacts and logs in: /artifacts/autoupgrade/run_1
    cluster.go:1849,autoupgrade.go:129,autoupgrade.go:262,test_runner.go:883: one or more parallel execution failure
        (1) attached stack trace
          -- stack trace:
          | github.com/cockroachdb/cockroach/pkg/roachprod/install.(*SyncedCluster).ParallelE
          |     github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:2202
          | github.com/cockroachdb/cockroach/pkg/roachprod/install.(*SyncedCluster).Parallel
          |     github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:2083
          | github.com/cockroachdb/cockroach/pkg/roachprod/install.(*SyncedCluster).Start
          |     github.com/cockroachdb/cockroach/pkg/roachprod/install/cockroach.go:175
          | github.com/cockroachdb/cockroach/pkg/roachprod.Start
          |     github.com/cockroachdb/cockroach/pkg/roachprod/roachprod.go:660
          | main.(*clusterImpl).StartE
          |     main/pkg/cmd/roachtest/cluster.go:1803
          | main.(*clusterImpl).Start
          |     main/pkg/cmd/roachtest/cluster.go:1848
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerAutoUpgrade.func1
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/autoupgrade.go:129
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerAutoUpgrade.func2
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/autoupgrade.go:262
          | main.(*testRunner).runTest.func2
          |     main/pkg/cmd/roachtest/test_runner.go:883
          | runtime.goexit
          |     GOROOT/src/runtime/asm_amd64.s:1581
        Wraps: (2) one or more parallel execution failure
        Error types: (1) *withstack.withStack (2) *errutil.leafError
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

/cc @cockroachdb/kv-triage

This test on roachdash | Improve this report!

Jira issue: CRDB-21972

aliher1911 commented 1 year ago

Superficially looks like an ssh failure / flake when roachtest calls cockroach version on remote starting new version after copying it over.

aliher1911 commented 1 year ago

There are other failures for test eng caused by the same recently ssh flakiness that they need to address so closing this off.