cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.1k stars 3.81k forks source link

roachtest: tpce/c=100000/nodes=5 failed #77398

Closed cockroach-teamcity closed 2 years ago

cockroach-teamcity commented 2 years ago

roachtest.tpce/c=100000/nodes=5 failed with artifacts on master @ 71eb44ffb1574350cf4a36143ed423f6ac0ca02b:

          | 5472ee6fba77: Pull complete
          | b4965326ef68: Pull complete
          | Digest: sha256:03e8e39d4adcd31d5d4fb059886c01122120528fc4ba757fbc6f7fdeea81632c
          | Status: Downloaded newer image for cockroachdb/tpc-e:latest
          | Error: Error { kind: Db, cause: Some(DbError { severity: "ERROR", parsed_severity: None, code: SqlState("XX000"), message: "internal error: Non-nullable column \"trade:t_dts\" with no value! Index scanned was \"trade_pkey\" with the index key columns (t_id) and the values (200000000149130)", detail: Some("stack trace:\ngithub.com/cockroachdb/cockroach/pkg/sql/row/fetcher.go:1106: finalizeRow()\ngithub.com/cockroachdb/cockroach/pkg/sql/row/fetcher.go:955: NextRow()\ngithub.com/cockroachdb/cockroach/pkg/sql/rowexec/tablereader.go:256: Next()\ngithub.com/cockroachdb/cockroach/pkg/sql/rowexec/sampler.go:253: mainLoop()\ngithub.com/cockroachdb/cockroach/pkg/sql/rowexec/sampler.go:229: Run()\ngithub.com/cockroachdb/cockroach/pkg/sql/flowinfra/flow.go:413: func1()\nGOROOT/src/runtime/asm_amd64.s:1581: goexit()\n"), hint: Some("You have encountered an unexpected error.\n\nPlease check the public issue tracker to check whether this problem is\nalready tracked. If you cannot find it there, please report the error\nwith details by creating a new issue.\n\nIf you would rather not post publicly, please contact us directly\nusing the support form.\n\nWe appreciate your feedback.\n"), position: None, where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("fetcher.go"), line: Some(1106), routine: Some("finalizeRow") }) }
          |
          | stdout:
          | Configuring cluster settings...
          | Initializing schema...
          | Importing dataset...
          | Collecting stats...
        Wraps: (4) COMMAND_PROBLEM
        Wraps: (5) Node 6. Command with error:
          | ``````
          | sudo docker run cockroachdb/tpc-e:latest --customers=100000 --racks=5 --init --hosts=10.128.0.49
          | ``````
        Wraps: (6) exit status 1
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) errors.Cmd (5) *hintdetail.withDetail (6) *exec.ExitError

    monitor.go:127,tpce.go:105,tpce.go:122,test_runner.go:866: monitor failure: monitor task failed: t.Fatal() was called
        (1) attached stack trace
          -- stack trace:
          | main.(*monitorImpl).WaitE
          |     main/pkg/cmd/roachtest/monitor.go:115
          | main.(*monitorImpl).Wait
          |     main/pkg/cmd/roachtest/monitor.go:123
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerTPCE.func1
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/tpce.go:105
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerTPCE.func2
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/tpce.go:122
          | main.(*testRunner).runTest.func2
          |     main/pkg/cmd/roachtest/test_runner.go:866
        Wraps: (2) monitor failure
        Wraps: (3) attached stack trace
          -- stack trace:
          | main.(*monitorImpl).wait.func2
          |     main/pkg/cmd/roachtest/monitor.go:171
        Wraps: (4) monitor task failed
        Wraps: (5) attached stack trace
          -- stack trace:
          | main.init
          |     main/pkg/cmd/roachtest/monitor.go:80
          | runtime.doInit
          |     GOROOT/src/runtime/proc.go:6498
          | runtime.main
          |     GOROOT/src/runtime/proc.go:238
          | runtime.goexit
          |     GOROOT/src/runtime/asm_amd64.s:1581
        Wraps: (6) t.Fatal() was called
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.leafError
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

/cc @cockroachdb/kv-triage

This test on roachdash | Improve this report!

Jira issue: CRDB-13571

blathers-crl[bot] commented 2 years ago

cc @cockroachdb/bulk-io

stevendanna commented 2 years ago

Should hopefully be fixed by https://github.com/cockroachdb/cockroach/pull/77403/commits/2a4f5190a636a91f3142dd81bee79df490740b8f

stevendanna commented 2 years ago

Close as fixed.