roachtest: admission-control/multitenant-fairness/write-heavy/even failed

cockroach-teamcity commented 3 months ago

roachtest.admission-control/multitenant-fairness/write-heavy/even failed with artifacts on master @ 8a97a5edd98336e2dd04ef12f08628fba84b17dd:

(assertions.go:363).Fail: 
    Error Trace:    github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/admission_control_multitenant_fairness.go:310
                                github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/admission_control_multitenant_fairness.go:103
                                main/pkg/cmd/roachtest/test_runner.go:1246
                                src/runtime/asm_amd64.s:1695
    Error:          Received unexpected error:
                    pq: internal error while retrieving user account memberships: operation "get-user-session" timed out after 10.001s (given timeout 10s): internal error while retrieving user account: get default settings error: interrupted during singleflight load-value:defaultsettings-roachprod-100-1: context deadline exceeded
    Test:           admission-control/multitenant-fairness/write-heavy/even
(require.go:1357).NoError: FailNow called
test artifacts and logs in: /artifacts/admission-control/multitenant-fairness/write-heavy/even/run_1

Parameters:

ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_fs=ext4
ROACHTEST_localSSD=true
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

/cc @cockroachdb/storage _{This test on roachdash | Improve this report!

Jira issue: CRDB-39737}

aadityasondhi commented 3 months ago

Seems like a timout on a sql query during the test setup. The USE kv call is timing out. We saw this earlier in https://github.com/cockroachdb/cockroach/issues/121404. Not a release blocker but not sure how to avoid this connection timeout.

cockroach-teamcity commented 1 month ago

roachtest.admission-control/multitenant-fairness/write-heavy/even failed with artifacts on master @ 356e1cc24a67d7e9a1e637b98d1664794f70fa03:

(assertions.go:363).Fail: 
    Error Trace:    github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/admission_control_multitenant_fairness.go:310
                                github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/admission_control_multitenant_fairness.go:103
                                main/pkg/cmd/roachtest/test_runner.go:1280
                                src/runtime/asm_amd64.s:1695
    Error:          Received unexpected error:
                    pq: internal error while retrieving user account memberships: operation "get-user-session" timed out after 10.001s (given timeout 10s): internal error while retrieving user account: get default settings error: interrupted during singleflight load-value:defaultsettings-roachprod-100-1: context deadline exceeded
    Test:           admission-control/multitenant-fairness/write-heavy/even
(require.go:1357).NoError: FailNow called
test artifacts and logs in: /artifacts/admission-control/multitenant-fairness/write-heavy/even/cpu_arch=arm64/run_1

Parameters:

ROACHTEST_arch=arm64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_fs=ext4
ROACHTEST_localSSD=false
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #128063 roachtest: admission-control/multitenant-fairness/write-heavy/even failed [C-test-failure O-roachtest O-robot branch-release-23.1 release-blocker]

This test on roachdash | Improve this report!

aadityasondhi commented 1 month ago

Should be fixed by https://github.com/cockroachdb/cockroach/pull/128394.

cockroachdb / cockroach

roachtest: admission-control/multitenant-fairness/write-heavy/even failed #126052