cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.21k stars 3.82k forks source link

roachtest: ycsb/F/nodes=3 failed #80057

Closed cockroach-teamcity closed 2 years ago

cockroach-teamcity commented 2 years ago

roachtest.ycsb/F/nodes=3 failed with artifacts on master @ 771432d1099e516dbc11827c5458886c176e73e3:

          | _elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
          |   211.0s        0         3473.1         3432.7      2.4     35.7     83.9    142.6 read
          |   211.0s        0         3481.1         3419.6     18.9     44.0     62.9    134.2 readModifyWrite
          |   212.0s        0         3536.4         3433.2      2.0     22.0    121.6    218.1 read
          |   212.0s        0         3688.4         3420.9     16.3     39.8     88.1    335.5 readModifyWrite
          |   213.0s        0         3563.7         3433.8      1.8     27.3    159.4    268.4 read
          |   213.0s        0         3623.6         3421.8     15.2     41.9    125.8    302.0 readModifyWrite
          |   214.0s        0          766.9         3421.3      2.4     21.0     56.6    142.6 read
          |   214.0s        0          773.9         3409.4     18.9     56.6     79.7    151.0 readModifyWrite
          |   215.0s        0            0.0         3405.4      0.0      0.0      0.0      0.0 read
          |   215.0s        0            0.0         3393.6      0.0      0.0      0.0      0.0 readModifyWrite
        Wraps: (4) COMMAND_PROBLEM
        Wraps: (5) Node 4. Command with error:
          | ``````
          | ./workload run ycsb --init --insert-count=1000000 --workload=F --concurrency=96 --splits=3 --histograms=perf/stats.json --select-for-update=true --ramp=2m --duration=30m {pgurl:1-3}
          | ``````
        Wraps: (6) exit status 1
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) errors.Cmd (5) *hintdetail.withDetail (6) *exec.ExitError

    monitor.go:127,ycsb.go:84,ycsb.go:101,test_runner.go:875: monitor failure: monitor task failed: t.Fatal() was called
        (1) attached stack trace
          -- stack trace:
          | main.(*monitorImpl).WaitE
          |     main/pkg/cmd/roachtest/monitor.go:115
          | main.(*monitorImpl).Wait
          |     main/pkg/cmd/roachtest/monitor.go:123
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerYCSB.func1
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/ycsb.go:84
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerYCSB.func2
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/ycsb.go:101
          | main.(*testRunner).runTest.func2
          |     main/pkg/cmd/roachtest/test_runner.go:875
        Wraps: (2) monitor failure
        Wraps: (3) attached stack trace
          -- stack trace:
          | main.(*monitorImpl).wait.func2
          |     main/pkg/cmd/roachtest/monitor.go:171
        Wraps: (4) monitor task failed
        Wraps: (5) attached stack trace
          -- stack trace:
          | main.init
          |     main/pkg/cmd/roachtest/monitor.go:80
          | runtime.doInit
          |     GOROOT/src/runtime/proc.go:6498
          | runtime.main
          |     GOROOT/src/runtime/proc.go:238
          | runtime.goexit
          |     GOROOT/src/runtime/asm_amd64.s:1581
        Wraps: (6) t.Fatal() was called
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

/cc @cockroachdb/kv-triage

This test on roachdash | Improve this report!

Jira issue: CRDB-15644

kvoli commented 2 years ago

tracking issue #80072, caused by #76609. Resolved in #80071.