Closed cockroach-teamcity closed 2 years ago
roachtest.zfs/ycsb/A/nodes=3/cpu=8 failed with artifacts on master @ 11e0a4da82124e70e772a009011ca7a4007bff85:
| (1) attached stack trace
| -- stack trace:
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*crdbInstallHelper).startNode
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:412
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.Cockroach.Start.func1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:166
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*SyncedCluster).ParallelE.func1.1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cluster_synced.go:1709
| | runtime.goexit
| | /usr/local/go/src/runtime/asm_amd64.s:1371
| Wraps: (2) ~ ./cockroach.sh
| | [0;1;31mJob for cockroach.service failed because the control process exited with error code.[0m
| | [0;1;31mSee "systemctl status cockroach.service" and "journalctl -xe" for details.[0m
| Wraps: (3) exit status 1
| Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *exec.ExitError:
| 1: ~ ./cockroach.sh: exit status 1
| (1) attached stack trace
| -- stack trace:
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*crdbInstallHelper).startNode
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:412
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.Cockroach.Start.func1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:166
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*SyncedCluster).ParallelE.func1.1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cluster_synced.go:1709
| | runtime.goexit
| | /usr/local/go/src/runtime/asm_amd64.s:1371
| Wraps: (2) ~ ./cockroach.sh
| | [0;1;31mJob for cockroach.service failed because the control process exited with error code.[0m
| | [0;1;31mSee "systemctl status cockroach.service" and "journalctl -xe" for details.[0m
| Wraps: (3) exit status 1
| Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *exec.ExitError:
| 2: ~ ./cockroach.sh: exit status 1
| (1) attached stack trace
| -- stack trace:
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*crdbInstallHelper).startNode
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:412
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.Cockroach.Start.func1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:166
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*SyncedCluster).ParallelE.func1.1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cluster_synced.go:1709
| | runtime.goexit
| | /usr/local/go/src/runtime/asm_amd64.s:1371
| Wraps: (2) ~ ./cockroach.sh
| | [0;1;31mJob for cockroach.service failed because the control process exited with error code.[0m
| | [0;1;31mSee "systemctl status cockroach.service" and "journalctl -xe" for details.[0m
| Wraps: (3) exit status 1
| Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *exec.ExitError:
| I210821 12:24:55.017794 1 (gostd) cluster_synced.go:1677 [-] 1 command failed
Wraps: (2) exit status 1
Error types: (1) *cluster.WithCommandDetails (2) *exec.ExitError
See: [roachtest README](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/roachtest) See: [CI job to stress roachtests](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestStress)
For the CI stress job, click the ellipsis (...) next to the Run button and fill in: * Changes / Build branch: master * Parameters / `env.TESTS`: `^zfs/ycsb/A/nodes=3/cpu=8$` * Parameters / `env.COUNT`: <number of runs>
/cc @cockroachdb/storage
roachtest.zfs/ycsb/A/nodes=3/cpu=8 failed with artifacts on master @ d18da6c092bf1522e7a6478fe3973817e318c247:
| (1) attached stack trace
| -- stack trace:
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*crdbInstallHelper).startNode
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:412
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.Cockroach.Start.func1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:166
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*SyncedCluster).ParallelE.func1.1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cluster_synced.go:1709
| | runtime.goexit
| | /usr/local/go/src/runtime/asm_amd64.s:1371
| Wraps: (2) ~ ./cockroach.sh
| | [0;1;31mJob for cockroach.service failed because the control process exited with error code.[0m
| | [0;1;31mSee "systemctl status cockroach.service" and "journalctl -xe" for details.[0m
| Wraps: (3) exit status 1
| Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *exec.ExitError:
| 1: ~ ./cockroach.sh: exit status 1
| (1) attached stack trace
| -- stack trace:
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*crdbInstallHelper).startNode
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:412
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.Cockroach.Start.func1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:166
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*SyncedCluster).ParallelE.func1.1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cluster_synced.go:1709
| | runtime.goexit
| | /usr/local/go/src/runtime/asm_amd64.s:1371
| Wraps: (2) ~ ./cockroach.sh
| | [0;1;31mJob for cockroach.service failed because the control process exited with error code.[0m
| | [0;1;31mSee "systemctl status cockroach.service" and "journalctl -xe" for details.[0m
| Wraps: (3) exit status 1
| Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *exec.ExitError:
| 2: ~ ./cockroach.sh: exit status 1
| (1) attached stack trace
| -- stack trace:
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*crdbInstallHelper).startNode
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:412
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.Cockroach.Start.func1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:166
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*SyncedCluster).ParallelE.func1.1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cluster_synced.go:1709
| | runtime.goexit
| | /usr/local/go/src/runtime/asm_amd64.s:1371
| Wraps: (2) ~ ./cockroach.sh
| | [0;1;31mJob for cockroach.service failed because the control process exited with error code.[0m
| | [0;1;31mSee "systemctl status cockroach.service" and "journalctl -xe" for details.[0m
| Wraps: (3) exit status 1
| Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *exec.ExitError:
| I210822 12:45:22.670547 1 (gostd) cluster_synced.go:1677 [-] 1 command failed
Wraps: (2) exit status 1
Error types: (1) *cluster.WithCommandDetails (2) *exec.ExitError
See: [roachtest README](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/roachtest) See: [CI job to stress roachtests](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestStress)
For the CI stress job, click the ellipsis (...) next to the Run button and fill in: * Changes / Build branch: master * Parameters / `env.TESTS`: `^zfs/ycsb/A/nodes=3/cpu=8$` * Parameters / `env.COUNT`: <number of runs>
/cc @cockroachdb/storage
roachtest.zfs/ycsb/A/nodes=3/cpu=8 failed with artifacts on master @ 61bd543ba7288c8f0eed6cddded7b219c9d1fcd4:
| (1) attached stack trace
| -- stack trace:
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*crdbInstallHelper).startNode
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:412
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.Cockroach.Start.func1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:166
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*SyncedCluster).ParallelE.func1.1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cluster_synced.go:1709
| | runtime.goexit
| | /usr/local/go/src/runtime/asm_amd64.s:1371
| Wraps: (2) ~ ./cockroach.sh
| | [0;1;31mJob for cockroach.service failed because the control process exited with error code.[0m
| | [0;1;31mSee "systemctl status cockroach.service" and "journalctl -xe" for details.[0m
| Wraps: (3) exit status 1
| Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *exec.ExitError:
| 1: ~ ./cockroach.sh: exit status 1
| (1) attached stack trace
| -- stack trace:
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*crdbInstallHelper).startNode
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:412
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.Cockroach.Start.func1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:166
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*SyncedCluster).ParallelE.func1.1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cluster_synced.go:1709
| | runtime.goexit
| | /usr/local/go/src/runtime/asm_amd64.s:1371
| Wraps: (2) ~ ./cockroach.sh
| | [0;1;31mJob for cockroach.service failed because the control process exited with error code.[0m
| | [0;1;31mSee "systemctl status cockroach.service" and "journalctl -xe" for details.[0m
| Wraps: (3) exit status 1
| Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *exec.ExitError:
| 2: ~ ./cockroach.sh: exit status 1
| (1) attached stack trace
| -- stack trace:
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*crdbInstallHelper).startNode
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:412
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.Cockroach.Start.func1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cockroach.go:166
| | github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install.(*SyncedCluster).ParallelE.func1.1
| | /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/install/cluster_synced.go:1709
| | runtime.goexit
| | /usr/local/go/src/runtime/asm_amd64.s:1371
| Wraps: (2) ~ ./cockroach.sh
| | [0;1;31mJob for cockroach.service failed because the control process exited with error code.[0m
| | [0;1;31mSee "systemctl status cockroach.service" and "journalctl -xe" for details.[0m
| Wraps: (3) exit status 1
| Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *exec.ExitError:
| I210823 12:29:54.274322 1 (gostd) cluster_synced.go:1677 [-] 1 command failed
Wraps: (2) exit status 1
Error types: (1) *cluster.WithCommandDetails (2) *exec.ExitError
See: [roachtest README](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/roachtest) See: [CI job to stress roachtests](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestStress)
For the CI stress job, click the ellipsis (...) next to the Run button and fill in: * Changes / Build branch: master * Parameters / `env.TESTS`: `^zfs/ycsb/A/nodes=3/cpu=8$` * Parameters / `env.COUNT`: <number of runs>
/cc @cockroachdb/storage
roachtest.ycsb/A/nodes=3 failed with artifacts on master @ 29716850b181718594663889ddb5f479fef7a305:
The test failed on branch=master, cloud=aws:
test artifacts and logs in: /artifacts/ycsb/A/nodes=3/run_1
cluster.go:1868,ycsb.go:62,ycsb.go:101,test_runner.go:875: one or more parallel execution failure
(1) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/roachprod/install.(*SyncedCluster).ParallelE
| github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:2042
| github.com/cockroachdb/cockroach/pkg/roachprod/install.(*SyncedCluster).Parallel
| github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:1923
| github.com/cockroachdb/cockroach/pkg/roachprod/install.(*SyncedCluster).Start
| github.com/cockroachdb/cockroach/pkg/roachprod/install/cockroach.go:167
| github.com/cockroachdb/cockroach/pkg/roachprod.Start
| github.com/cockroachdb/cockroach/pkg/roachprod/roachprod.go:660
| main.(*clusterImpl).StartE
| main/pkg/cmd/roachtest/cluster.go:1826
| main.(*clusterImpl).Start
| main/pkg/cmd/roachtest/cluster.go:1867
| github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerYCSB.func1
| github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/ycsb.go:62
| github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerYCSB.func2
| github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/ycsb.go:101
| main.(*testRunner).runTest.func2
| main/pkg/cmd/roachtest/test_runner.go:875
| runtime.goexit
| GOROOT/src/runtime/asm_amd64.s:1581
Wraps: (2) one or more parallel execution failure
Error types: (1) *withstack.withStack (2) *errutil.leafError
See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)
roachtest.ycsb/A/nodes=3 failed with artifacts on master @ 29716850b181718594663889ddb5f479fef7a305:
The test failed on branch=master, cloud=gce:
test artifacts and logs in: /artifacts/ycsb/A/nodes=3/run_1
cluster.go:1868,ycsb.go:62,ycsb.go:101,test_runner.go:875: one or more parallel execution failure
(1) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/roachprod/install.(*SyncedCluster).ParallelE
| github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:2042
| github.com/cockroachdb/cockroach/pkg/roachprod/install.(*SyncedCluster).Parallel
| github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:1923
| github.com/cockroachdb/cockroach/pkg/roachprod/install.(*SyncedCluster).Start
| github.com/cockroachdb/cockroach/pkg/roachprod/install/cockroach.go:167
| github.com/cockroachdb/cockroach/pkg/roachprod.Start
| github.com/cockroachdb/cockroach/pkg/roachprod/roachprod.go:660
| main.(*clusterImpl).StartE
| main/pkg/cmd/roachtest/cluster.go:1826
| main.(*clusterImpl).Start
| main/pkg/cmd/roachtest/cluster.go:1867
| github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerYCSB.func1
| github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/ycsb.go:62
| github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerYCSB.func2
| github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/ycsb.go:101
| main.(*testRunner).runTest.func2
| main/pkg/cmd/roachtest/test_runner.go:875
| runtime.goexit
| GOROOT/src/runtime/asm_amd64.s:1581
Wraps: (2) one or more parallel execution failure
Error types: (1) *withstack.withStack (2) *errutil.leafError
See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)
This looks like a false alarm:
Wraps: (2) ~ ./cockroach version --build-tag
| panic: file does not exist
|
| goroutine 1 [running]:
| github.com/cockroachdb/cockroach/pkg/ui/distccl.init.0()
| github.com/cockroachdb/cockroach/pkg/ui/distccl/distccl.go:40 +0x125
Closing.
roachtest.zfs/ycsb/A/nodes=3/cpu=8 failed with artifacts on master @ 7897f24246bef3cb94f9f4bfaed474ecaa9fdee6:
Reproduce
See: [roachtest README](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/roachtest) See: [CI job to stress roachtests](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Nightlies_RoachtestStress)
For the CI stress job, click the ellipsis (...) next to the Run button and fill in: * Changes / Build branch: master * Parameters / `env.TESTS`: `^zfs/ycsb/A/nodes=3/cpu=8$` * Parameters / `env.COUNT`: <number of runs>
/cc @cockroachdb/storage
This test on roachdash | Improve this report!
Jira issue: CRDB-9485