cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.1k stars 3.81k forks source link

ccl/changefeedccl: TestTableNameCollision failed #128868

Open cockroach-teamcity opened 2 months ago

cockroach-teamcity commented 2 months ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ 4c8ca4cf727684f1f2acd54da6ff4cf60fdf6587:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 2080935 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe2db20?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc00efe9000)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d6a738, 0xc006a43d60})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d6a738, 0xc006a43d60}, 0x2, 0x3, 0x0, 0x3?, {0x66aa1b4, 0x21}, {0xc00454dc70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc01d5128c0, {0x7d6a738?, 0xc006a43d60})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d6a738?, 0xc006a43d60?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 2080864
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision2618105839 test_log_scope.go:81: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:862: making server as system tenant helpers_test.go:937: making kafka feed factory testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/cdc

This test on roachdash | Improve this report!

Jira issue: CRDB-41241

cockroach-teamcity commented 2 months ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ 9e7ed1f7ee5346ea2357389f06963e411fd49306:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 1930082 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe33be0?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc00c7bc400)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d6ea38, 0xc00e803e50})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d6ea38, 0xc00e803e50}, 0x2, 0x3, 0x0, 0x3?, {0x66ade6b, 0x21}, {0xc00577bc70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc0188c5960, {0x7d6ea38?, 0xc00e803e50})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d6ea38?, 0xc00e803e50?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1929869
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision2893917735 test_log_scope.go:81: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:862: making server as system tenant helpers_test.go:937: making kafka feed factory ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

andyyang890 commented 2 months ago

I'm suspecting it might be related to the new Kafka sink given the stack trace and also the metamorphic constant:

initialized metamorphic constant "changefeed.new_kafka_sink.enabled" with value true

Assigning Miles to investigate.

cockroach-teamcity commented 2 months ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ b5a9e987af5f8b8eabbaa31db2b19ec270099d0f:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 2019574 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe33be0?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc00a2efc00)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d6eab8, 0xc009737a40})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d6eab8, 0xc009737a40}, 0x2, 0x3, 0x0, 0x3?, {0x66adeeb, 0x21}, {0xc008cbdc70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc011053b20, {0x7d6eab8?, 0xc009737a40})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d6eab8?, 0xc009737a40?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 2019547
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision2271642091 test_log_scope.go:81: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:862: making server as system tenant helpers_test.go:937: making kafka feed factory ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #129212 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc branch-release-23.2.10-rc release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 months ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ c59b0c5754de22b5c395c56eb1bed8f5d4b20bee:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 2128895 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe33be0?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc0069fe400)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d6ec98, 0xc017f9e8c0})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d6ec98, 0xc017f9e8c0}, 0x2, 0x3, 0x0, 0x3?, {0x66adfeb, 0x21}, {0xc0040e1c70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc03649c1c0, {0x7d6ec98?, 0xc017f9e8c0})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d6ec98?, 0xc017f9e8c0?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 2128884
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision3623741478 test_log_scope.go:81: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:862: making server as system tenant helpers_test.go:937: making kafka feed factory testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #129212 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc branch-release-23.2.10-rc release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 months ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ 44dbe07325a795f1b723762e162d095a2e2246ca:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 1918639 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe34c20?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc00bd4e800)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d6f738, 0xc00c917b30})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d6f738, 0xc00c917b30}, 0x2, 0x3, 0x0, 0x3?, {0x66ae9d4, 0x21}, {0xc0054c7c70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc008ede460, {0x7d6f738?, 0xc00c917b30})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d6f738?, 0xc00c917b30?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1918614
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision897003722 test_log_scope.go:81: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:862: making server as system tenant helpers_test.go:937: making kafka feed factory ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #129212 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc branch-release-23.2.10-rc release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 months ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ e711c1e222715d1de6434f1b725c2ae7a3c5c096:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 2078383 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe38c60?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc00f135000)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d72118, 0xc01546a690})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d72118, 0xc01546a690}, 0x2, 0x3, 0x0, 0x3?, {0x66b09af, 0x21}, {0xc002ad1c70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc0145da540, {0x7d72118?, 0xc01546a690})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d72118?, 0xc01546a690?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 2078359
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision2150457677 test_log_scope.go:81: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:862: making server as system tenant helpers_test.go:937: making kafka feed factory ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #129212 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc branch-release-23.2.10-rc release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 months ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ 547d7ea96aca72ba7f68616e308b1981d6dac76c:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 2174795 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe38c60?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc00aca5000)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d72218, 0xc00715ba90})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d72218, 0xc00715ba90}, 0x2, 0x3, 0x0, 0x3?, {0x66b0b19, 0x21}, {0xc0035cfc70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc00ee56c40, {0x7d72218?, 0xc00715ba90})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d72218?, 0xc00715ba90?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 2174701
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision1863230480 test_log_scope.go:81: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:862: making server as system tenant helpers_test.go:937: making kafka feed factory testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #129212 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc branch-release-23.2.10-rc release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 months ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ 547d7ea96aca72ba7f68616e308b1981d6dac76c:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 2090021 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe38c60?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc0070bc800)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d72218, 0xc00991d680})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d72218, 0xc00991d680}, 0x2, 0x3, 0x0, 0x3?, {0x66b0b19, 0x21}, {0xc008c23c70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc00bbadce0, {0x7d72218?, 0xc00991d680})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d72218?, 0xc00991d680?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 2090000
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision1422750006 test_log_scope.go:81: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:862: making server as system tenant helpers_test.go:937: making kafka feed factory ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #129212 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc branch-release-23.2.10-rc release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 months ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ b658f8a1a5c7924c1879fe66bb87bab2ce46e789:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 1988825 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe38c60?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc00e494c00)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d72458, 0xc00dd540f0})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d72458, 0xc00dd540f0}, 0x2, 0x3, 0x0, 0x3?, {0x66b0c99, 0x21}, {0xc004577c70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc012adf7a0, {0x7d72458?, 0xc00dd540f0})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d72458?, 0xc00dd540f0?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1988763
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision210265965 test_log_scope.go:81: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:862: making server as system tenant helpers_test.go:937: making kafka feed factory testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #129212 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot P-2 T-cdc branch-release-23.2.10-rc release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 month ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ a1a1bf97dd897f3f93b08ac8382bca2af2fd04d7:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 1941965 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe66ca0?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc011815800)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7d8d568, 0xc009d66d70})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7d8d568, 0xc009d66d70}, 0x2, 0x3, 0x0, 0x3?, {0x66c873a, 0x21}, {0xc00599dc70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc00c09b180, {0x7d8d568?, 0xc009d66d70})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7d8d568?, 0xc009d66d70?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1941934
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision1106226578 test_log_scope.go:81: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:862: making server as system tenant helpers_test.go:937: making kafka feed factory testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #130465 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc X-duplicate branch-release-23.2.11-rc release-blocker] - #129212 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot P-2 T-cdc branch-release-23.2.10-rc release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 month ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ 52d7f3db720359153214894c6dd3b4f96bd8be6b:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 2007671 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe85ee0?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc00b478c00)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7da1068, 0xc005906c80})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7da1068, 0xc005906c80}, 0x2, 0x3, 0x0, 0x3?, {0x66d969c, 0x21}, {0xc0030d9c70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc0045375e0, {0x7da1068?, 0xc005906c80})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7da1068?, 0xc005906c80?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 2007634
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision1772713820 test_log_scope.go:81: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:862: making server as system tenant helpers_test.go:937: making kafka feed factory ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #130465 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc X-duplicate branch-release-23.2.11-rc release-blocker] - #129212 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot P-2 T-cdc branch-release-23.2.10-rc release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 month ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ e1a7cbe2adde82bdbe67c46237f005158c5ecf82:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 2051709 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe86ee0?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc0092f0400)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7da11e8, 0xc021601450})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7da11e8, 0xc021601450}, 0x2, 0x3, 0x0, 0x3?, {0x66d979c, 0x21}, {0xc00548dc70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc01c15da40, {0x7da11e8?, 0xc021601450})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7da11e8?, 0xc021601450?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 2051669
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision2758389029 test_log_scope.go:81: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:862: making server as system tenant helpers_test.go:937: making kafka feed factory testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #130465 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc X-duplicate branch-release-23.2.11-rc release-blocker] - #129212 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot P-2 T-cdc branch-release-23.2.10-rc release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 month ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ 5fbe8f17051cfe89d38bfab3e370e0e89919e34a:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 2032286 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe86ee0?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc013a6d000)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7da11e8, 0xc00f1fe370})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7da11e8, 0xc00f1fe370}, 0x2, 0x3, 0x0, 0x3?, {0x66d979c, 0x21}, {0xc00abd1c70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc00fbde380, {0x7da11e8?, 0xc00f1fe370})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7da11e8?, 0xc00f1fe370?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 2032233
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision619709202 test_log_scope.go:81: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:862: making server as system tenant helpers_test.go:937: making kafka feed factory ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #130465 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc X-duplicate branch-release-23.2.11-rc release-blocker] - #129212 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot P-2 T-cdc branch-release-23.2.10-rc release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 month ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ 5ae18a722f6efad6b8943d2514aed72a6b2e920a:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 1964910 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe87ee0?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc00b24c000)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7da2008, 0xc014877b80})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7da2008, 0xc014877b80}, 0x2, 0x3, 0x0, 0x3?, {0x66da43c, 0x21}, {0xc00a085c70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc00a37a8c0, {0x7da2008?, 0xc014877b80})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7da2008?, 0xc014877b80?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1964887
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision3050486831 test_log_scope.go:81: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:862: making server as system tenant helpers_test.go:937: making kafka feed factory testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #130465 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc X-duplicate branch-release-23.2.11-rc release-blocker] - #129212 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot P-2 T-cdc branch-release-23.2.10-rc release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 month ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ 021942d1a7a71bd30fe3aefff6f6a4470cee2a0e:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 2032238 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe8cf20?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc01a0a5000)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7da4d48, 0xc0219d6f00})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7da4d48, 0xc0219d6f00}, 0x2, 0x3, 0x0, 0x3?, {0x66dc722, 0x21}, {0xc00548bc70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc02689a2a0, {0x7da4d48?, 0xc0219d6f00})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7da4d48?, 0xc0219d6f00?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 2032160
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision2458699170 test_log_scope.go:81: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:862: making server as system tenant helpers_test.go:937: making kafka feed factory testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection testfeed_test.go:303: creating external connection testfeed_test.go:306: ran create external connection ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #131067 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc X-duplicate branch-release-23.2.12-rc] - #130465 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc X-duplicate branch-release-23.2.11-rc] - #129212 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot P-2 T-cdc branch-release-23.2.10-rc release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 month ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ 021942d1a7a71bd30fe3aefff6f6a4470cee2a0e:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: <stack not captured. Set debugUseAfterFinish>

Stack:

goroutine 2069223 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe8cf20?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc01520c400)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7da4d48, 0xc00fc97310})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:92 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7da4d48, 0xc00fc97310}, 0x2, 0x3, 0x0, 0x3?, {0x66dc722, 0x21}, {0xc0057b7c70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:104 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:39
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc0195517a0, {0x7da4d48?, 0xc00fc97310})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:429 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7da4d48?, 0xc00fc97310?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:539 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:168 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 2069162
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision2480905453 test_log_scope.go:81: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:862: making server as system tenant helpers_test.go:937: making kafka feed factory ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #131067 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc X-duplicate branch-release-23.2.12-rc] - #130465 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot T-cdc X-duplicate branch-release-23.2.11-rc] - #129212 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot P-2 T-cdc branch-release-23.2.10-rc release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 weeks ago

ccl/changefeedccl.TestTableNameCollision failed with artifacts on release-23.2 @ e93a9dfcf3cc37657a48992aedb47f5684534651:

Fatal error:

panic: use of Span after Finish. Span: changefntr. Finish previously called at: goroutine 2134039 [running]:
runtime/debug.Stack()
    GOROOT/src/runtime/debug/stack.go:24 +0x5e
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).finishInternal(0xc011841000)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:251 +0x67
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).Finish(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:242
github.com/cockroachdb/cockroach/pkg/sql/execinfra.(*ProcessorBaseNoHelper).InternalClose(0xc00edcf000)
    github.com/cockroachdb/cockroach/pkg/sql/execinfra/processorsbase.go:918 +0xb5
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*changeFrontier).close(0xc00edcf000)
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_processors.go:1434 +0x45
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*changeFrontier).ConsumerClosed(0xc005f78000?)
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_processors.go:1904 +0x13
github.com/cockroachdb/cockroach/pkg/sql/execinfra.Run({0x7da1938, 0xc009dd11d0}, {0x7dc3080, 0xc00edcf000}, {0x7d6aea0?, 0xc005f78000?})
    github.com/cockroachdb/cockroach/pkg/sql/execinfra/base.go:204 +0xa4
github.com/cockroachdb/cockroach/pkg/sql/execinfra.(*ProcessorBaseNoHelper).Run(0xc00edcf000, {0x7da1900?, 0xc00e92bbd0?}, {0x7d6aea0?, 0xc005f78000})
    github.com/cockroachdb/cockroach/pkg/sql/execinfra/processorsbase.go:721 +0x6d
github.com/cockroachdb/cockroach/pkg/sql/flowinfra.(*FlowBase).Run(0xc003414c80, {0x7da1900?, 0xc00e92bbd0}, 0x0?)
    github.com/cockroachdb/cockroach/pkg/sql/flowinfra/flow.go:574 +0x222
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run(0xc00f2e2c80, {0x7da1900, 0xc00e92bb80}, 0xc00c466f00, 0x0, 0xc0083f2d80, 0xc005f78000, 0xc00edce800, 0xc015aaff18)
    github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:914 +0xb9e
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.startDistChangefeed.func1({0x7da1900, 0xc00e92bae0})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_dist.go:316 +0x408
github.com/cockroachdb/cockroach/pkg/util/ctxgroup.GoAndWait.Group.GoCtx.func1()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:163 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 2133789
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96

Stack:

goroutine 2134142 [running]:
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish(0xbe7b360?)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:177 +0x114
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType(0xc011841000)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:437 +0x18
github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose(...)
    github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:445
github.com/cockroachdb/cockroach/pkg/util/log.getSpanOrEventLog({0x7da1900, 0xc01c376f50})
    github.com/cockroachdb/cockroach/pkg/util/log/trace.go:87 +0x34
github.com/cockroachdb/cockroach/pkg/util/log.logfDepthInternal({0x7da1900, 0xc01c376f50}, 0x2, 0x3, 0x0, 0x3?, {0x66d637b, 0x21}, {0xc0197ffc70, 0x1, ...})
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:99 +0x4b4
github.com/cockroachdb/cockroach/pkg/util/log.logfDepth(...)
    github.com/cockroachdb/cockroach/pkg/util/log/channels.go:34
github.com/cockroachdb/cockroach/pkg/util/log.Errorf(...)
    github.com/cockroachdb/cockroach/bazel-out/k8-fastbuild/bin/pkg/util/log/log_channels_generated.go:654
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.(*batchingSink).runBatchingWorker(0xc01bb5ae00, {0x7da1900?, 0xc01c376f50})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:431 +0x5af
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.func1({0x7da1900?, 0xc01c376f50?})
    github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/batching_sink.go:541 +0x2a
github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.makeBatchingSink.Group.GoCtx.func2()
    github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:163 +0x22
golang.org/x/sync/errgroup.(*Group).Go.func1()
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78 +0x56
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 2134039
    golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:75 +0x96
Log preceding fatal error

``` === RUN TestTableNameCollision test_log_scope.go:165: test logs captured to: /artifacts/tmp/_tmp/905465f697dc04c39f7d8a82cb057bfe/logTestTableNameCollision1934911423 test_log_scope.go:76: use -show-logs to present logs inline === RUN TestTableNameCollision/kafka helpers_test.go:873: making server as system tenant helpers_test.go:948: making kafka feed factory testfeed_test.go:300: creating external connection testfeed_test.go:303: ran create external connection testfeed_test.go:300: creating external connection testfeed_test.go:303: ran create external connection testfeed_test.go:300: creating external connection testfeed_test.go:303: ran create external connection ```

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #131067 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot P-2 T-cdc X-duplicate branch-release-23.2.12-rc] - #130465 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot P-2 T-cdc X-duplicate branch-release-23.2.11-rc] - #129212 ccl/changefeedccl: TestTableNameCollision failed [A-cdc C-test-failure O-robot P-2 T-cdc branch-release-23.2.10-rc release-blocker]

This test on roachdash | Improve this report!