search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.96k stars 3.79k forks source link

pkg/server/storage_api/storage_api_test: TestAdminDecommissionedOperations failed #131296

Open cockroach-teamcity opened 1 week ago

cockroach-teamcity commented 1 week ago

pkg/server/storage_api/storage_api_test.TestAdminDecommissionedOperations failed with artifacts on release-23.2.12-rc @ c3ddfa7a1b062e6829c74dbf4067ef0479092b36:

=== RUN   TestAdminDecommissionedOperations
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/800529510c8a77f305555fe4848c57f0/logTestAdminDecommissionedOperations1287146211
    test_log_scope.go:81: use -show-logs to present logs inline
    test_server_shim.go:93: cluster virtualization disabled due to issue: #81590 (expected label: C-bug)

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:819: (TestAdminDecommissionedOperations)
    NOTICE: .NodeID() called via implicit interface StorageLayerInterface;
HINT: consider using .StorageLayer().NodeID() instead.

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:827: (TestAdminDecommissionedOperations)
    NOTICE: .NodeID() called via implicit interface StorageLayerInterface;
HINT: consider using .StorageLayer().NodeID() instead.

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:833: (TestAdminDecommissionedOperations.func1)
    NOTICE: .DB() called via implicit interface ApplicationLayerInterface;
HINT: consider using .ApplicationLayer().DB() instead.
TIP: consider replacing the test server initialization from:
    ts, ... := serverutils.StartServer(t, ...)
    defer ts.Stopper().Stop(...)
to:
    srv, ... := serverutils.StartServer(t, ...)
    defer srv.Stopper().Stop(...)
    ts := srv.ApplicationLayer()

See also: https://go.crdb.dev/p/testserver-and-cluster-virtualization
    decommission_test.go:977: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/800529510c8a77f305555fe4848c57f0/logTestAdminDecommissionedOperations1287146211
--- FAIL: TestAdminDecommissionedOperations (9.82s)
=== RUN   TestAdminDecommissionedOperations/Events
    decommission_test.go:972: 
            Error Trace:    pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:972
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:75
                                        github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:213
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:81
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:60
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:42
                                        pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:954
            Error:          Not equal: 
                            expected: 0xd
                            actual  : 0x7
            Test:           TestAdminDecommissionedOperations/Events
            Messages:       rpc error: code = PermissionDenied desc = admin-events: RangeIterator failed to seek to /Table/12/2: failed to connect to n1 at 127.0.0.1:41323: initial connection heartbeat failed: grpc: n2 was permanently removed from the cluster at 2024-09-24 15:52:40.502405858 +0000 UTC; it is not allowed to rejoin the cluster [code 7/PermissionDenied], node is decommissioned
    --- FAIL: TestAdminDecommissionedOperations/Events (0.01s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/unowned

This test on roachdash | Improve this report!

Jira issue: CRDB-42486

cockroach-teamcity commented 1 week ago

pkg/server/storage_api/storage_api_test.TestAdminDecommissionedOperations failed with artifacts on release-23.2.12-rc @ c3ddfa7a1b062e6829c74dbf4067ef0479092b36:

=== RUN   TestAdminDecommissionedOperations
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/800529510c8a77f305555fe4848c57f0/logTestAdminDecommissionedOperations1140480251
    test_log_scope.go:81: use -show-logs to present logs inline
    test_server_shim.go:93: cluster virtualization disabled due to issue: #81590 (expected label: C-bug)

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:819: (TestAdminDecommissionedOperations)
    NOTICE: .NodeID() called via implicit interface StorageLayerInterface;
HINT: consider using .StorageLayer().NodeID() instead.

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:827: (TestAdminDecommissionedOperations)
    NOTICE: .NodeID() called via implicit interface StorageLayerInterface;
HINT: consider using .StorageLayer().NodeID() instead.

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:833: (TestAdminDecommissionedOperations.func1)
    NOTICE: .DB() called via implicit interface ApplicationLayerInterface;
HINT: consider using .ApplicationLayer().DB() instead.
TIP: consider replacing the test server initialization from:
    ts, ... := serverutils.StartServer(t, ...)
    defer ts.Stopper().Stop(...)
to:
    srv, ... := serverutils.StartServer(t, ...)
    defer srv.Stopper().Stop(...)
    ts := srv.ApplicationLayer()

See also: https://go.crdb.dev/p/testserver-and-cluster-virtualization
    decommission_test.go:977: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/800529510c8a77f305555fe4848c57f0/logTestAdminDecommissionedOperations1140480251
--- FAIL: TestAdminDecommissionedOperations (8.06s)
=== RUN   TestAdminDecommissionedOperations/Events
    decommission_test.go:972: 
            Error Trace:    pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:972
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:75
                                        github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:213
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:81
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:60
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:42
                                        pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:954
            Error:          Not equal: 
                            expected: 0xd
                            actual  : 0x7
            Test:           TestAdminDecommissionedOperations/Events
            Messages:       rpc error: code = PermissionDenied desc = admin-events: RangeIterator failed to seek to /Table/12/2: failed to connect to n1 at 127.0.0.1:41449: initial connection heartbeat failed: grpc: n2 was permanently removed from the cluster at 2024-09-25 15:49:13.043025222 +0000 UTC; it is not allowed to rejoin the cluster [code 7/PermissionDenied], node is decommissioned
    --- FAIL: TestAdminDecommissionedOperations/Events (0.01s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

pkg/server/storage_api/storage_api_test.TestAdminDecommissionedOperations failed with artifacts on release-23.2.12-rc @ c3ddfa7a1b062e6829c74dbf4067ef0479092b36:

=== RUN   TestAdminDecommissionedOperations
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/800529510c8a77f305555fe4848c57f0/logTestAdminDecommissionedOperations4035702370
    test_log_scope.go:81: use -show-logs to present logs inline
    test_server_shim.go:93: cluster virtualization disabled due to issue: #81590 (expected label: C-bug)

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:819: (TestAdminDecommissionedOperations)
    NOTICE: .NodeID() called via implicit interface StorageLayerInterface;
HINT: consider using .StorageLayer().NodeID() instead.

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:827: (TestAdminDecommissionedOperations)
    NOTICE: .NodeID() called via implicit interface StorageLayerInterface;
HINT: consider using .StorageLayer().NodeID() instead.

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:833: (TestAdminDecommissionedOperations.func1)
    NOTICE: .DB() called via implicit interface ApplicationLayerInterface;
HINT: consider using .ApplicationLayer().DB() instead.
TIP: consider replacing the test server initialization from:
    ts, ... := serverutils.StartServer(t, ...)
    defer ts.Stopper().Stop(...)
to:
    srv, ... := serverutils.StartServer(t, ...)
    defer srv.Stopper().Stop(...)
    ts := srv.ApplicationLayer()

See also: https://go.crdb.dev/p/testserver-and-cluster-virtualization
    decommission_test.go:977: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/800529510c8a77f305555fe4848c57f0/logTestAdminDecommissionedOperations4035702370
--- FAIL: TestAdminDecommissionedOperations (9.32s)
=== RUN   TestAdminDecommissionedOperations/Events
    decommission_test.go:972: 
            Error Trace:    pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:972
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:75
                                        github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:213
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:81
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:60
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:42
                                        pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:954
            Error:          Not equal: 
                            expected: 0xd
                            actual  : 0x7
            Test:           TestAdminDecommissionedOperations/Events
            Messages:       rpc error: code = PermissionDenied desc = admin-events: RangeIterator failed to seek to /Table/12/2: failed to connect to n1 at 127.0.0.1:39999: initial connection heartbeat failed: grpc: n2 was permanently removed from the cluster at 2024-09-26 16:02:32.345585511 +0000 UTC; it is not allowed to rejoin the cluster [code 7/PermissionDenied], node is decommissioned
    --- FAIL: TestAdminDecommissionedOperations/Events (0.01s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #131403 pkg/server/storage_api/storage_api_test: TestAdminDecommissionedOperations failed [C-test-failure O-robot branch-release-24.1 release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

pkg/server/storage_api/storage_api_test.TestAdminDecommissionedOperations failed with artifacts on release-23.2.12-rc @ c3ddfa7a1b062e6829c74dbf4067ef0479092b36:

=== RUN   TestAdminDecommissionedOperations
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/800529510c8a77f305555fe4848c57f0/logTestAdminDecommissionedOperations527856779
    test_log_scope.go:81: use -show-logs to present logs inline
    test_server_shim.go:93: cluster virtualization disabled due to issue: #81590 (expected label: C-bug)

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:819: (TestAdminDecommissionedOperations)
    NOTICE: .NodeID() called via implicit interface StorageLayerInterface;
HINT: consider using .StorageLayer().NodeID() instead.

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:827: (TestAdminDecommissionedOperations)
    NOTICE: .NodeID() called via implicit interface StorageLayerInterface;
HINT: consider using .StorageLayer().NodeID() instead.

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:833: (TestAdminDecommissionedOperations.func1)
    NOTICE: .DB() called via implicit interface ApplicationLayerInterface;
HINT: consider using .ApplicationLayer().DB() instead.
TIP: consider replacing the test server initialization from:
    ts, ... := serverutils.StartServer(t, ...)
    defer ts.Stopper().Stop(...)
to:
    srv, ... := serverutils.StartServer(t, ...)
    defer srv.Stopper().Stop(...)
    ts := srv.ApplicationLayer()

See also: https://go.crdb.dev/p/testserver-and-cluster-virtualization
    decommission_test.go:977: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/800529510c8a77f305555fe4848c57f0/logTestAdminDecommissionedOperations527856779
--- FAIL: TestAdminDecommissionedOperations (3.46s)
=== RUN   TestAdminDecommissionedOperations/Events
    decommission_test.go:972: 
            Error Trace:    pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:972
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:75
                                        github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:213
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:81
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:60
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:42
                                        pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:954
            Error:          Not equal: 
                            expected: 0xd
                            actual  : 0x7
            Test:           TestAdminDecommissionedOperations/Events
            Messages:       rpc error: code = PermissionDenied desc = admin-events: RangeIterator failed to seek to /Table/12/2: failed to connect to n1 at 127.0.0.1:45481: initial connection heartbeat failed: grpc: n2 was permanently removed from the cluster at 2024-09-26 16:09:43.016210204 +0000 UTC; it is not allowed to rejoin the cluster [code 7/PermissionDenied], node is decommissioned
    --- FAIL: TestAdminDecommissionedOperations/Events (0.00s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #131403 pkg/server/storage_api/storage_api_test: TestAdminDecommissionedOperations failed [C-test-failure O-robot branch-release-24.1 release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 6 days ago

pkg/server/storage_api/storage_api_test.TestAdminDecommissionedOperations failed with artifacts on release-23.2.12-rc @ c3ddfa7a1b062e6829c74dbf4067ef0479092b36:

=== RUN   TestAdminDecommissionedOperations
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/800529510c8a77f305555fe4848c57f0/logTestAdminDecommissionedOperations2667521697
    test_log_scope.go:81: use -show-logs to present logs inline
    test_server_shim.go:93: cluster virtualization disabled due to issue: #81590 (expected label: C-bug)

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:819: (TestAdminDecommissionedOperations)
    NOTICE: .NodeID() called via implicit interface StorageLayerInterface;
HINT: consider using .StorageLayer().NodeID() instead.

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:827: (TestAdminDecommissionedOperations)
    NOTICE: .NodeID() called via implicit interface StorageLayerInterface;
HINT: consider using .StorageLayer().NodeID() instead.

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:833: (TestAdminDecommissionedOperations.func1)
    NOTICE: .DB() called via implicit interface ApplicationLayerInterface;
HINT: consider using .ApplicationLayer().DB() instead.
TIP: consider replacing the test server initialization from:
    ts, ... := serverutils.StartServer(t, ...)
    defer ts.Stopper().Stop(...)
to:
    srv, ... := serverutils.StartServer(t, ...)
    defer srv.Stopper().Stop(...)
    ts := srv.ApplicationLayer()

See also: https://go.crdb.dev/p/testserver-and-cluster-virtualization
    decommission_test.go:977: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/800529510c8a77f305555fe4848c57f0/logTestAdminDecommissionedOperations2667521697
--- FAIL: TestAdminDecommissionedOperations (8.64s)
=== RUN   TestAdminDecommissionedOperations/Events
    decommission_test.go:972: 
            Error Trace:    pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:972
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:75
                                        github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:213
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:81
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:60
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:42
                                        pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:954
            Error:          Not equal: 
                            expected: 0xd
                            actual  : 0x7
            Test:           TestAdminDecommissionedOperations/Events
            Messages:       rpc error: code = PermissionDenied desc = admin-events: RangeIterator failed to seek to /Table/12/2: failed to connect to n1 at 127.0.0.1:37927: initial connection heartbeat failed: grpc: n2 was permanently removed from the cluster at 2024-09-27 15:57:28.229701204 +0000 UTC; it is not allowed to rejoin the cluster [code 7/PermissionDenied], node is decommissioned
    --- FAIL: TestAdminDecommissionedOperations/Events (0.01s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #131403 pkg/server/storage_api/storage_api_test: TestAdminDecommissionedOperations failed [C-test-failure O-robot branch-release-24.1 release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 6 days ago

pkg/server/storage_api/storage_api_test.TestAdminDecommissionedOperations failed with artifacts on release-23.2.12-rc @ c3ddfa7a1b062e6829c74dbf4067ef0479092b36:

=== RUN   TestAdminDecommissionedOperations
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/800529510c8a77f305555fe4848c57f0/logTestAdminDecommissionedOperations2115865312
    test_log_scope.go:81: use -show-logs to present logs inline
    test_server_shim.go:93: cluster virtualization disabled due to issue: #81590 (expected label: C-bug)

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:819: (TestAdminDecommissionedOperations)
    NOTICE: .NodeID() called via implicit interface StorageLayerInterface;
HINT: consider using .StorageLayer().NodeID() instead.

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:827: (TestAdminDecommissionedOperations)
    NOTICE: .NodeID() called via implicit interface StorageLayerInterface;
HINT: consider using .StorageLayer().NodeID() instead.

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:833: (TestAdminDecommissionedOperations.func1)
    NOTICE: .DB() called via implicit interface ApplicationLayerInterface;
HINT: consider using .ApplicationLayer().DB() instead.
TIP: consider replacing the test server initialization from:
    ts, ... := serverutils.StartServer(t, ...)
    defer ts.Stopper().Stop(...)
to:
    srv, ... := serverutils.StartServer(t, ...)
    defer srv.Stopper().Stop(...)
    ts := srv.ApplicationLayer()

See also: https://go.crdb.dev/p/testserver-and-cluster-virtualization
    decommission_test.go:977: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/800529510c8a77f305555fe4848c57f0/logTestAdminDecommissionedOperations2115865312
--- FAIL: TestAdminDecommissionedOperations (2.92s)
=== RUN   TestAdminDecommissionedOperations/Drain
    decommission_test.go:972: 
            Error Trace:    pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:972
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:75
                                        github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:213
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:81
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:60
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:42
                                        pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:954
            Error:          Not equal: 
                            expected: 0x7
                            actual  : 0x2
            Test:           TestAdminDecommissionedOperations/Drain
            Messages:       rpc error: code = Unknown desc = some sessions did not respond to cancellation within 1s
    --- FAIL: TestAdminDecommissionedOperations/Drain (1.00s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #131403 pkg/server/storage_api/storage_api_test: TestAdminDecommissionedOperations failed [C-test-failure O-robot branch-release-24.1 release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 5 days ago

pkg/server/storage_api/storage_api_test.TestAdminDecommissionedOperations failed with artifacts on release-23.2.12-rc @ c3ddfa7a1b062e6829c74dbf4067ef0479092b36:

=== RUN   TestAdminDecommissionedOperations
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/800529510c8a77f305555fe4848c57f0/logTestAdminDecommissionedOperations2762792803
    test_log_scope.go:81: use -show-logs to present logs inline
    test_server_shim.go:93: cluster virtualization disabled due to issue: #81590 (expected label: C-bug)

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:819: (TestAdminDecommissionedOperations)
    NOTICE: .NodeID() called via implicit interface StorageLayerInterface;
HINT: consider using .StorageLayer().NodeID() instead.

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:827: (TestAdminDecommissionedOperations)
    NOTICE: .NodeID() called via implicit interface StorageLayerInterface;
HINT: consider using .StorageLayer().NodeID() instead.

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:833: (TestAdminDecommissionedOperations.func1)
    NOTICE: .DB() called via implicit interface ApplicationLayerInterface;
HINT: consider using .ApplicationLayer().DB() instead.
TIP: consider replacing the test server initialization from:
    ts, ... := serverutils.StartServer(t, ...)
    defer ts.Stopper().Stop(...)
to:
    srv, ... := serverutils.StartServer(t, ...)
    defer srv.Stopper().Stop(...)
    ts := srv.ApplicationLayer()

See also: https://go.crdb.dev/p/testserver-and-cluster-virtualization
    decommission_test.go:977: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/800529510c8a77f305555fe4848c57f0/logTestAdminDecommissionedOperations2762792803
--- FAIL: TestAdminDecommissionedOperations (8.22s)
=== RUN   TestAdminDecommissionedOperations/Events
    decommission_test.go:972: 
            Error Trace:    pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:972
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:75
                                        github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:213
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:81
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:60
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:42
                                        pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:954
            Error:          Not equal: 
                            expected: 0xd
                            actual  : 0x7
            Test:           TestAdminDecommissionedOperations/Events
            Messages:       rpc error: code = PermissionDenied desc = admin-events: RangeIterator failed to seek to /Table/12/2: failed to connect to n1 at 127.0.0.1:44627: initial connection heartbeat failed: grpc: n2 was permanently removed from the cluster at 2024-09-28 15:54:58.909224625 +0000 UTC; it is not allowed to rejoin the cluster [code 7/PermissionDenied], node is decommissioned
    --- FAIL: TestAdminDecommissionedOperations/Events (0.01s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #131403 pkg/server/storage_api/storage_api_test: TestAdminDecommissionedOperations failed [C-test-failure O-robot branch-release-24.1 release-blocker]

This test on roachdash | Improve this report!

cockroach-teamcity commented 4 days ago

pkg/server/storage_api/storage_api_test.TestAdminDecommissionedOperations failed with artifacts on release-23.2.12-rc @ c3ddfa7a1b062e6829c74dbf4067ef0479092b36:

=== RUN   TestAdminDecommissionedOperations
    test_log_scope.go:170: test logs captured to: /artifacts/tmp/_tmp/800529510c8a77f305555fe4848c57f0/logTestAdminDecommissionedOperations2063264782
    test_log_scope.go:81: use -show-logs to present logs inline
    test_server_shim.go:93: cluster virtualization disabled due to issue: #81590 (expected label: C-bug)

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:819: (TestAdminDecommissionedOperations)
    NOTICE: .NodeID() called via implicit interface StorageLayerInterface;
HINT: consider using .StorageLayer().NodeID() instead.

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:827: (TestAdminDecommissionedOperations)
    NOTICE: .NodeID() called via implicit interface StorageLayerInterface;
HINT: consider using .StorageLayer().NodeID() instead.

pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:833: (TestAdminDecommissionedOperations.func1)
    NOTICE: .DB() called via implicit interface ApplicationLayerInterface;
HINT: consider using .ApplicationLayer().DB() instead.
TIP: consider replacing the test server initialization from:
    ts, ... := serverutils.StartServer(t, ...)
    defer ts.Stopper().Stop(...)
to:
    srv, ... := serverutils.StartServer(t, ...)
    defer srv.Stopper().Stop(...)
    ts := srv.ApplicationLayer()

See also: https://go.crdb.dev/p/testserver-and-cluster-virtualization
    decommission_test.go:977: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/800529510c8a77f305555fe4848c57f0/logTestAdminDecommissionedOperations2063264782
--- FAIL: TestAdminDecommissionedOperations (8.31s)
=== RUN   TestAdminDecommissionedOperations/Events
    decommission_test.go:972: 
            Error Trace:    pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:972
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:75
                                        github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:213
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:81
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:60
                                        github.com/cockroachdb/cockroach/pkg/testutils/soon.go:42
                                        pkg/server/storage_api/storage_api_test_test/pkg/server/storage_api/decommission_test.go:954
            Error:          Not equal: 
                            expected: 0xd
                            actual  : 0x7
            Test:           TestAdminDecommissionedOperations/Events
            Messages:       rpc error: code = PermissionDenied desc = admin-events: RangeIterator failed to seek to /Table/12/2: failed to connect to n1 at 127.0.0.1:45857: initial connection heartbeat failed: grpc: n2 was permanently removed from the cluster at 2024-09-29 16:06:20.952682207 +0000 UTC; it is not allowed to rejoin the cluster [code 7/PermissionDenied], node is decommissioned
    --- FAIL: TestAdminDecommissionedOperations/Events (0.01s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #131403 pkg/server/storage_api/storage_api_test: TestAdminDecommissionedOperations failed [C-test-failure O-robot branch-release-24.1 release-blocker]

This test on roachdash | Improve this report!