search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.2k stars 3.82k forks source link

roachtest: cdc/sink-chaos failed #80903

Closed cockroach-teamcity closed 1 year ago

cockroach-teamcity commented 2 years ago

roachtest.cdc/sink-chaos failed with artifacts on release-22.1 @ 8e92d074c6efe338f83a3307aa3363d4750f005e:

          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.kafkaManager.installJRE
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:1274
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.kafkaManager.install
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:1263
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.cdcBasicTest
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:159
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerCDC.func4
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:743
          | main.(*testRunner).runTest.func2
          |     main/pkg/cmd/roachtest/test_runner.go:875
          | runtime.goexit
          |     GOROOT/src/runtime/asm_amd64.s:1581
        Wraps: (2) output in run_064510.346634270_n4_sudo
        Wraps: (3) sudo DEBIAN_FRONTEND=noninteractive apt-get -yq --no-install-recommends install openssl default-jre 2>&1 > logs/apt-get-install.log returned
          | stderr:
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | buntu4_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/p/pcsc-lite/libpcsclite1_1.8.26-3_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/j/java-common/default-jre-headless_1.11-72_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/c/ca-certificates-java/ca-certificates-java_20190405ubuntu1_all.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/libx/libxcb/libxcb-dri2-0_1.14-2_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/libx/libxcb/libxcb-dri3-0_1.14-2_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/libx/libxcb/libxcb-glx0_1.14-2_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/libx/libxcb/libxcb-present0_1.14-2_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/libx/libxcb/libxcb-shm0_1.14-2_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/libx/libxcb/libxcb-sync1_1.14-2_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/libx/libxcb/libxcb-xfixes0_1.14-2_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/libx/libxfixes/libxfixes3_5.0.3-2_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/libx/libxshmfence/libxshmfence1_1.3-1_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/libx/libxxf86vm/libxxf86vm1_1.1.4-1build1_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/libp/libpciaccess/libpciaccess0_0.16-0ubuntu1_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/l/lm-sensors/libsensors-config_3.6.0-2ubuntu1_all.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/l/lm-sensors/libsensors5_3.6.0-2ubuntu1_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/v/vulkan-loader/libvulkan1_1.2.131.2-1_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/g/giflib/libgif7_5.1.9-1_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/libx/libxi/libxi6_1.7.10-0ubuntu1_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/libx/libxrender/libxrender1_0.9.10-1_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/x/xorg/x11-common_7.7+19ubuntu14_all.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/libx/libxtst/libxtst6_1.2.3-1_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Failed to fetch http://us-east1.gce.archive.ubuntu.com/ubuntu/pool/main/j/java-common/default-jre_1.11-72_amd64.deb  503  Service Unavailable [IP: 35.196.128.168 80]
          | E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
        Wraps: (4) COMMAND_PROBLEM
        Wraps: (5) Node 4. Command with error:
          | ``````
          | sudo DEBIAN_FRONTEND=noninteractive apt-get -yq --no-install-recommends install openssl default-jre 2>&1 > logs/apt-get-install.log
          | ``````
        Wraps: (6) exit status 100
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) errors.Cmd (5) *hintdetail.withDetail (6) *exec.ExitError
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

/cc @cockroachdb/cdc

This test on roachdash | Improve this report!

Jira issue: CRDB-15460

Epic CRDB-11732

HonoreDB commented 2 years ago

This looks like not just a flake, but a flake related to setting up Kafka which we only do in tests, so I'm removing the release-blocker tag.

cockroach-teamcity commented 1 year ago

roachtest.cdc/sink-chaos failed with artifacts on release-22.1 @ 000c9624b56b09d5fbd06557c559b2f910142a9c:

          |   |     github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:2083
          |   | github.com/cockroachdb/cockroach/pkg/roachprod/install.(*SyncedCluster).Run
          |   |     github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:661
          |   | github.com/cockroachdb/cockroach/pkg/roachprod.Run
          |   |     github.com/cockroachdb/cockroach/pkg/roachprod/roachprod.go:384
          |   | main.execCmdEx
          |   |     main/pkg/cmd/roachtest/cluster.go:341
          |   | main.execCmd
          |   |     main/pkg/cmd/roachtest/cluster.go:229
          |   | main.(*clusterImpl).RunE
          |   |     main/pkg/cmd/roachtest/cluster.go:1954
          |   | main.(*clusterImpl).Run
          |   |     main/pkg/cmd/roachtest/cluster.go:1932
          |   | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tpccWorkload).run
          |   |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:1578
          |   | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.cdcBasicTest.func1
          |   |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:186
          |   | main.(*monitorImpl).Go.func1
          |   |     main/pkg/cmd/roachtest/monitor.go:105
          |   | golang.org/x/sync/errgroup.(*Group).Go.func1
          |   |     golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:57
          |   | runtime.goexit
          |   |     GOROOT/src/runtime/asm_amd64.s:1581
          | Wraps: (2) one or more parallel execution failure
          | Error types: (1) *withstack.withStack (2) *errutil.leafError
        Wraps: (5) context canceled
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *secondary.withSecondaryError (5) *errors.errorString

    monitor.go:127,cdc.go:296,cdc.go:743,test_runner.go:883: monitor failure: monitor task failed: pq: Use of CHANGEFEED requires an enterprise license. Your evaluation license expired on December 30, 2022. If you're interested in getting a new license, please contact subscriptions@cockroachlabs.com and we can help you out.
        (1) attached stack trace
          -- stack trace:
          | main.(*monitorImpl).WaitE
          |     main/pkg/cmd/roachtest/monitor.go:115
          | main.(*monitorImpl).Wait
          |     main/pkg/cmd/roachtest/monitor.go:123
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.cdcBasicTest
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:296
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerCDC.func4
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:743
          | [...repeated from below...]
        Wraps: (2) monitor failure
        Wraps: (3) attached stack trace
          -- stack trace:
          | main.(*monitorImpl).wait.func2
          |     main/pkg/cmd/roachtest/monitor.go:171
          | runtime.goexit
          |     GOROOT/src/runtime/asm_amd64.s:1581
        Wraps: (4) monitor task failed
        Wraps: (5) pq: Use of CHANGEFEED requires an enterprise license. Your evaluation license expired on December 30, 2022. If you're interested in getting a new license, please contact subscriptions@cockroachlabs.com and we can help you out.
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *pq.Error
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #93781 roachtest: cdc/sink-chaos failed [C-test-failure O-roachtest O-robot T-cdc branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cdc/sink-chaos failed with artifacts on release-22.1 @ 000c9624b56b09d5fbd06557c559b2f910142a9c:

          |   |     github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:2083
          |   | github.com/cockroachdb/cockroach/pkg/roachprod/install.(*SyncedCluster).Run
          |   |     github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:661
          |   | github.com/cockroachdb/cockroach/pkg/roachprod.Run
          |   |     github.com/cockroachdb/cockroach/pkg/roachprod/roachprod.go:384
          |   | main.execCmdEx
          |   |     main/pkg/cmd/roachtest/cluster.go:341
          |   | main.execCmd
          |   |     main/pkg/cmd/roachtest/cluster.go:229
          |   | main.(*clusterImpl).RunE
          |   |     main/pkg/cmd/roachtest/cluster.go:1954
          |   | main.(*clusterImpl).Run
          |   |     main/pkg/cmd/roachtest/cluster.go:1932
          |   | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tpccWorkload).run
          |   |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:1578
          |   | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.cdcBasicTest.func1
          |   |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:186
          |   | main.(*monitorImpl).Go.func1
          |   |     main/pkg/cmd/roachtest/monitor.go:105
          |   | golang.org/x/sync/errgroup.(*Group).Go.func1
          |   |     golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:57
          |   | runtime.goexit
          |   |     GOROOT/src/runtime/asm_amd64.s:1581
          | Wraps: (2) one or more parallel execution failure
          | Error types: (1) *withstack.withStack (2) *errutil.leafError
        Wraps: (5) context canceled
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *secondary.withSecondaryError (5) *errors.errorString

    monitor.go:127,cdc.go:296,cdc.go:743,test_runner.go:883: monitor failure: monitor task failed: pq: Use of CHANGEFEED requires an enterprise license. Your evaluation license expired on December 30, 2022. If you're interested in getting a new license, please contact subscriptions@cockroachlabs.com and we can help you out.
        (1) attached stack trace
          -- stack trace:
          | main.(*monitorImpl).WaitE
          |     main/pkg/cmd/roachtest/monitor.go:115
          | main.(*monitorImpl).Wait
          |     main/pkg/cmd/roachtest/monitor.go:123
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.cdcBasicTest
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:296
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerCDC.func4
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:743
          | [...repeated from below...]
        Wraps: (2) monitor failure
        Wraps: (3) attached stack trace
          -- stack trace:
          | main.(*monitorImpl).wait.func2
          |     main/pkg/cmd/roachtest/monitor.go:171
          | runtime.goexit
          |     GOROOT/src/runtime/asm_amd64.s:1581
        Wraps: (4) monitor task failed
        Wraps: (5) pq: Use of CHANGEFEED requires an enterprise license. Your evaluation license expired on December 30, 2022. If you're interested in getting a new license, please contact subscriptions@cockroachlabs.com and we can help you out.
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *pq.Error
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #94503 roachtest: cdc/sink-chaos failed [C-test-failure O-roachtest O-robot T-cdc branch-master release-blocker] - #93781 roachtest: cdc/sink-chaos failed [C-test-failure O-roachtest O-robot T-cdc branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cdc/sink-chaos failed with artifacts on release-22.1 @ 000c9624b56b09d5fbd06557c559b2f910142a9c:

          |   |     github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:2083
          |   | github.com/cockroachdb/cockroach/pkg/roachprod/install.(*SyncedCluster).Run
          |   |     github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:661
          |   | github.com/cockroachdb/cockroach/pkg/roachprod.Run
          |   |     github.com/cockroachdb/cockroach/pkg/roachprod/roachprod.go:384
          |   | main.execCmdEx
          |   |     main/pkg/cmd/roachtest/cluster.go:341
          |   | main.execCmd
          |   |     main/pkg/cmd/roachtest/cluster.go:229
          |   | main.(*clusterImpl).RunE
          |   |     main/pkg/cmd/roachtest/cluster.go:1954
          |   | main.(*clusterImpl).Run
          |   |     main/pkg/cmd/roachtest/cluster.go:1932
          |   | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tpccWorkload).run
          |   |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:1578
          |   | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.cdcBasicTest.func1
          |   |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:186
          |   | main.(*monitorImpl).Go.func1
          |   |     main/pkg/cmd/roachtest/monitor.go:105
          |   | golang.org/x/sync/errgroup.(*Group).Go.func1
          |   |     golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:57
          |   | runtime.goexit
          |   |     GOROOT/src/runtime/asm_amd64.s:1581
          | Wraps: (2) one or more parallel execution failure
          | Error types: (1) *withstack.withStack (2) *errutil.leafError
        Wraps: (5) context canceled
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *secondary.withSecondaryError (5) *errors.errorString

    monitor.go:127,cdc.go:296,cdc.go:743,test_runner.go:883: monitor failure: monitor task failed: pq: Use of CHANGEFEED requires an enterprise license. Your evaluation license expired on December 30, 2022. If you're interested in getting a new license, please contact subscriptions@cockroachlabs.com and we can help you out.
        (1) attached stack trace
          -- stack trace:
          | main.(*monitorImpl).WaitE
          |     main/pkg/cmd/roachtest/monitor.go:115
          | main.(*monitorImpl).Wait
          |     main/pkg/cmd/roachtest/monitor.go:123
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.cdcBasicTest
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:296
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerCDC.func4
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:743
          | [...repeated from below...]
        Wraps: (2) monitor failure
        Wraps: (3) attached stack trace
          -- stack trace:
          | main.(*monitorImpl).wait.func2
          |     main/pkg/cmd/roachtest/monitor.go:171
          | runtime.goexit
          |     GOROOT/src/runtime/asm_amd64.s:1581
        Wraps: (4) monitor task failed
        Wraps: (5) pq: Use of CHANGEFEED requires an enterprise license. Your evaluation license expired on December 30, 2022. If you're interested in getting a new license, please contact subscriptions@cockroachlabs.com and we can help you out.
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *pq.Error
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #94503 roachtest: cdc/sink-chaos failed [C-test-failure O-roachtest O-robot T-cdc branch-master release-blocker] - #93781 roachtest: cdc/sink-chaos failed [C-test-failure O-roachtest O-robot T-cdc branch-release-22.2]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 year ago

roachtest.cdc/sink-chaos failed with artifacts on release-22.1 @ 000c9624b56b09d5fbd06557c559b2f910142a9c:

          |   |     github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:2083
          |   | github.com/cockroachdb/cockroach/pkg/roachprod/install.(*SyncedCluster).Run
          |   |     github.com/cockroachdb/cockroach/pkg/roachprod/install/cluster_synced.go:661
          |   | github.com/cockroachdb/cockroach/pkg/roachprod.Run
          |   |     github.com/cockroachdb/cockroach/pkg/roachprod/roachprod.go:384
          |   | main.execCmdEx
          |   |     main/pkg/cmd/roachtest/cluster.go:341
          |   | main.execCmd
          |   |     main/pkg/cmd/roachtest/cluster.go:229
          |   | main.(*clusterImpl).RunE
          |   |     main/pkg/cmd/roachtest/cluster.go:1954
          |   | main.(*clusterImpl).Run
          |   |     main/pkg/cmd/roachtest/cluster.go:1932
          |   | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tpccWorkload).run
          |   |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:1578
          |   | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.cdcBasicTest.func1
          |   |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:186
          |   | main.(*monitorImpl).Go.func1
          |   |     main/pkg/cmd/roachtest/monitor.go:105
          |   | golang.org/x/sync/errgroup.(*Group).Go.func1
          |   |     golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:57
          |   | runtime.goexit
          |   |     GOROOT/src/runtime/asm_amd64.s:1581
          | Wraps: (2) one or more parallel execution failure
          | Error types: (1) *withstack.withStack (2) *errutil.leafError
        Wraps: (5) context canceled
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *secondary.withSecondaryError (5) *errors.errorString

    monitor.go:127,cdc.go:296,cdc.go:743,test_runner.go:883: monitor failure: monitor task failed: pq: Use of CHANGEFEED requires an enterprise license. Your evaluation license expired on December 30, 2022. If you're interested in getting a new license, please contact subscriptions@cockroachlabs.com and we can help you out.
        (1) attached stack trace
          -- stack trace:
          | main.(*monitorImpl).WaitE
          |     main/pkg/cmd/roachtest/monitor.go:115
          | main.(*monitorImpl).Wait
          |     main/pkg/cmd/roachtest/monitor.go:123
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.cdcBasicTest
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:296
          | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerCDC.func4
          |     github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/cdc.go:743
          | [...repeated from below...]
        Wraps: (2) monitor failure
        Wraps: (3) attached stack trace
          -- stack trace:
          | main.(*monitorImpl).wait.func2
          |     main/pkg/cmd/roachtest/monitor.go:171
          | runtime.goexit
          |     GOROOT/src/runtime/asm_amd64.s:1581
        Wraps: (4) monitor task failed
        Wraps: (5) pq: Use of CHANGEFEED requires an enterprise license. Your evaluation license expired on December 30, 2022. If you're interested in getting a new license, please contact subscriptions@cockroachlabs.com and we can help you out.
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *pq.Error
Help

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md) See: [How To Investigate \(internal\)](https://cockroachlabs.atlassian.net/l/c/SSSBr8c7)

Same failure on other branches

- #94503 roachtest: cdc/sink-chaos failed [C-test-failure O-roachtest O-robot T-cdc branch-master release-blocker] - #93781 roachtest: cdc/sink-chaos failed [C-test-failure O-roachtest O-robot T-cdc branch-release-22.2]

This test on roachdash | Improve this report!

miretskiy commented 1 year ago

Moving back to triage -- this issue keeps happening

amruss commented 1 year ago

This one can be closed - license expired error