search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.07k stars 3.8k forks source link

sql/catalog/tabledesc: TestRemoveDefaultExprFromComputedColumn failed [timed out while starting tenant] #115345

Closed cockroach-teamcity closed 10 months ago

cockroach-teamcity commented 11 months ago

sql/catalog/tabledesc.TestRemoveDefaultExprFromComputedColumn failed with artifacts on release-23.2 @ a82f4141adcdc63605e1c6f79957d938a2676de0:

Fatal error:

panic: test timed out after 55s
running tests:
    TestRemoveDefaultExprFromComputedColumn (11s)

Stack: 

goroutine 35671 [running]:
testing.(*M).startAlarm.func1()
    GOROOT/src/testing/testing.go:2241 +0x31c
created by time.goFunc
    GOROOT/src/time/sleep.go:176 +0x38
Log preceding fatal error

``` === RUN TestRemoveDefaultExprFromComputedColumn test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/7b4578e3c0124849ee4088d817d512f1/logTestRemoveDefaultExprFromComputedColumn186734491 test_log_scope.go:81: use -show-logs to present logs inline test_server_shim.go:159: automatically injected a shared process virtual cluster under test; see comment at top of test_server_shim.go for details. ```

Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)

Same failure on other branches

- #114210 sql/catalog/tabledesc: TestRemoveDefaultExprFromComputedColumn failed [timed out while starting tenant] [C-test-failure O-robot T-multitenant branch-master release-blocker]

/cc @cockroachdb/sql-foundations

This test on roachdash | Improve this report!

Jira issue: CRDB-33990

rafiss commented 11 months ago

I see this goroutine:

goroutine 30800 [select]:
github.com/cockroachdb/cockroach/pkg/server.(*serverController).startAndWaitForRunningServer(0x4003aad340, {0x5c9d9e8, 0x40001b2000}, {0x48a538d?, 0x1?})
    github.com/cockroachdb/cockroach/pkg/server/server_controller_orchestration.go:235 +0xcc
github.com/cockroachdb/cockroach/pkg/server.(*testServer).StartSharedProcessTenant(_, {_, _}, {{0x48a538d, 0xb}, {0xa}, {{0x0, 0x0}, {0x0, 0x0}, ...}, ...})
    github.com/cockroachdb/cockroach/pkg/server/testserver.go:1385 +0x4cc
github.com/cockroachdb/cockroach/pkg/server.(*testServer).startSharedProcessDefaultTestTenant(0x4000b21560?, {0x5c9d9e8, 0x40001b2000})
    github.com/cockroachdb/cockroach/pkg/server/testserver.go:667 +0x78
github.com/cockroachdb/cockroach/pkg/server.(*testServer).maybeStartDefaultTestTenant(0x4003810000, {0x5c9d9e8?, 0x40001b2000?})
    github.com/cockroachdb/cockroach/pkg/server/testserver.go:717 +0x25c
github.com/cockroachdb/cockroach/pkg/server.(*testServer).Activate(0x4003810000, {0x5c9d9e8, 0x40001b2000})
    github.com/cockroachdb/cockroach/pkg/server/testserver.go:844 +0xbc
github.com/cockroachdb/cockroach/pkg/server.(*testServer).Start(0x4003810000, {0x5c9d9e8, 0x40001b2000})
    github.com/cockroachdb/cockroach/pkg/server/testserver.go:886 +0xa4
github.com/cockroachdb/cockroach/pkg/testutils/serverutils.(*wrap).Start(0x0?, {0x5c9d9e8, 0x40001b2000})
    github.com/cockroachdb/cockroach/bazel-out/aarch64-fastbuild/bin/pkg/testutils/serverutils/ts_control_forwarder_generated.go:15 +0x54
github.com/cockroachdb/cockroach/pkg/testutils/serverutils.StartServerOnlyE({_, _}, {{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...})
    github.com/cockroachdb/cockroach/pkg/testutils/serverutils/test_server_shim.go:279 +0x188
github.com/cockroachdb/cockroach/pkg/testutils/serverutils.StartServerOnly({_, _}, {{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...})
    github.com/cockroachdb/cockroach/pkg/testutils/serverutils/test_server_shim.go:294 +0x6c
github.com/cockroachdb/cockroach/pkg/testutils/serverutils.StartServer({_, _}, {{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...})
    github.com/cockroachdb/cockroach/pkg/testutils/serverutils/test_server_shim.go:312 +0x50
github.com/cockroachdb/cockroach/pkg/sql/catalog/tabledesc_test.TestRemoveDefaultExprFromComputedColumn(0x4004e52340)
    github.com/cockroachdb/cockroach/pkg/sql/catalog/tabledesc_test/pkg/sql/catalog/tabledesc/structured_test.go:1014 +0xc8
testing.tRunner(0x4004e52340, 0x4bdb348)
    GOROOT/src/testing/testing.go:1576 +0x104
created by testing.(*T).Run
    GOROOT/src/testing/testing.go:1629 +0x370

This appears similar to https://github.com/cockroachdb/cockroach/issues/114210

stevendanna commented 10 months ago

I'm going to close this for now as I've merged a fix that I hope addresses a good portion of some shared-process test-server slowness. I'm sure there is more to handle but I don't see evidence that it would be particular to this test.

stevendanna commented 10 months ago

Oops, missed the branch. We can close this once we merge the backport for my fix.

yuzefovich commented 10 months ago

@stevendanna you're thinking about backport of #115246, right?

stevendanna commented 10 months ago

Closing this. Let's re-open if it re-occurs.