cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.07k stars 3.8k forks source link

roachtest: jepsen/g2/majority-ring-start-kill-2 failed #68863

Closed cockroach-teamcity closed 3 years ago

cockroach-teamcity commented 3 years ago

roachtest.jepsen/g2/majority-ring-start-kill-2 failed with artifacts on master @ e723a85739a2e1b514f5de3221a2177a3a37d0e6:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/jepsen/g2/majority-ring-start-kill-2/run_1
    cluster.go:1887,jepsen.go:93,jepsen.go:147,jepsen.go:360,test_runner.go:777: output in run_182628.256138265_n1-6_sh: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-3290921-1628748769-66-n6cpu4:1-6 -- sh -c "sudo apt-get -y upgrade -o Dpkg::Options::='--force-confold' > logs/apt-upgrade.log 2>&1" returned: context canceled
        (1) attached stack trace
          -- stack trace:
          | main.(*clusterImpl).RunE
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1977
          | main.(*clusterImpl).Run
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1885
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.initJepsen
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/jepsen.go:93
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runJepsen
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/jepsen.go:147
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.RegisterJepsen.func1
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/jepsen.go:360
          | main.(*testRunner).runTest.func2
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:777
          | runtime.goexit
          |     /usr/local/go/src/runtime/asm_amd64.s:1371
        Wraps: (2) output in run_182628.256138265_n1-6_sh
        Wraps: (3) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-3290921-1628748769-66-n6cpu4:1-6 -- sh -c "sudo apt-get -y upgrade -o Dpkg::Options::='--force-confold' > logs/apt-upgrade.log 2>&1" returned
          | stderr:
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
        Wraps: (4) secondary error attachment
          | signal: killed
          | (1) signal: killed
          | Error types: (1) *exec.ExitError
        Wraps: (5) context canceled
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *secondary.withSecondaryError (5) *errors.errorString
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/roachtest) See: [CI job to stress roachtests](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestStress)

For the CI stress job, click the ellipsis (...) next to the Run button and fill in: * Changes / Build branch: master * Parameters / `env.TESTS`: `^jepsen/g2/majority-ring-start-kill-2$` * Parameters / `env.COUNT`: <number of runs>

Same failure on other branches

- #68845 roachtest: jepsen/g2/majority-ring-start-kill-2 failed [C-test-failure O-roachtest O-robot branch-release-21.1 release-blocker]

/cc @cockroachdb/kv-triage

This test on roachdash | Improve this report!

erikgrinaker commented 3 years ago

68794