search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.01k stars 3.79k forks source link

bench/rttanalysis: TestBenchmarkExpectation failed #131134

Open cockroach-teamcity opened 3 weeks ago

cockroach-teamcity commented 3 weeks ago

bench/rttanalysis.TestBenchmarkExpectation failed with artifacts on master @ c43628ced71b1089367e302f9aba0417baa6f43f:

    validate_benchmark_data.go:125: success: expected ORMQueries/hasura_column_descriptions to perform KV lookups in [5, 5], got 6
    validate_benchmark_data.go:125: success: expected AlterTableAddColumn/alter_table_add_1_column to perform KV lookups in [15, 15], got 15
    validate_benchmark_data.go:125: success: expected Revoke/revoke_all_on_1_table to perform KV lookups in [10, 10], got 10
    validate_benchmark_data.go:125: success: expected ORMQueries/has_sequence_privilege to perform KV lookups in [0, 0], got 0
    validate_benchmark_data.go:125: success: expected Revoke/revoke_all_on_2_tables to perform KV lookups in [10, 10], got 10
    validate_benchmark_data.go:125: success: expected AlterTableAddColumn/alter_table_add_2_columns to perform KV lookups in [15, 15], got 15
    validate_benchmark_data.go:125: success: expected DropRole/drop_1_role to perform KV lookups in [34, 34], got 34
    validate_benchmark_data.go:125: success: expected Jobs/show_job to perform KV lookups in [3, 3], got 3
    validate_benchmark_data.go:125: success: expected ORMQueries/hasura_column_descriptions_8_tables to perform KV lookups in [20, 20], got 20
    validate_benchmark_data.go:125: success: expected Jobs/pause_job to perform KV lookups in [8, 8], got 8
    validate_benchmark_data.go:125: success: expected GenerateObjects/generate_1000_tables_-_this_test_should_use_the_same_number_of_RTTs_as_for_10_tables to perform KV lookups in [5, 5], got 5
    validate_benchmark_data.go:125: success: expected Jobs/cancel_job to perform KV lookups in [6, 6], got 6
    validate_benchmark_data.go:125: success: expected ORMQueries/activerecord_type_introspection_query to perform KV lookups in [3, 3], got 3
    validate_benchmark_data.go:125: success: expected ORMQueries/hasura_column_descriptions_modified to perform KV lookups in [5, 5], got 5
    validate_benchmark_data.go:125: success: expected Audit/select_from_an_audit_table to perform KV lookups in [5, 5], got 6
    validate_benchmark_data.go:125: success: expected GenerateObjects/generate_50000_tables to perform KV lookups in [5, 5], got 5
    validate_benchmark_data.go:125: success: expected Jobs/jobs_page_type_filtered to perform KV lookups in [3, 3], got 4
    validate_benchmark_data.go:125: success: expected DropRole/drop_2_roles to perform KV lookups in [43, 43], got 43
    validate_benchmark_data.go:125: success: expected Jobs/show_jobs to perform KV lookups in [3, 5], got 3
    validate_benchmark_data.go:125: success: expected ORMQueries/liquibase_migrations to perform KV lookups in [4, 4], got 4
    validate_benchmark_data.go:122: fail: expected Jobs/jobs_page_latest_50 to perform KV lookups in [3, 3], got 5
    validate_benchmark_data.go:125: success: expected ORMQueries/pg_my_temp_schema_multiple_times to perform KV lookups in [3, 3], got 3
    validate_benchmark_data.go:125: success: expected GenerateObjects/generate_100_tables_from_template to perform KV lookups in [11, 11], got 12
    validate_benchmark_data.go:125: success: expected GenerateObjects/generate_10_tables to perform KV lookups in [5, 5], got 5
    validate_benchmark_data.go:125: success: expected ORMQueries/pg_attribute to perform KV lookups in [4, 4], got 4
    validate_benchmark_data.go:125: success: expected AlterTableAddCheckConstraint/alter_table_add_3_check_constraints to perform KV lookups in [15, 15], got 15
    validate_benchmark_data.go:125: success: expected AlterTableAddCheckConstraint/alter_table_add_2_check_constraints to perform KV lookups in [15, 15], got 15
    validate_benchmark_data.go:125: success: expected ORMQueries/prisma_types_4 to perform KV lookups in [5, 5], got 5
    validate_benchmark_data.go:125: success: expected AlterTableAddCheckConstraint/alter_table_add_1_check_constraint to perform KV lookups in [15, 15], got 15
    validate_benchmark_data.go:125: success: expected ORMQueries/prisma_column_descriptions_updated to perform KV lookups in [3, 3], got 4
    validate_benchmark_data.go:125: success: expected ORMQueries/prisma_types_16 to perform KV lookups in [5, 5], got 5
    validate_benchmark_data.go:125: success: expected Jobs/non_admin_show_jobs to perform KV lookups in [4, 4], got 4
    validate_benchmark_data.go:125: success: expected Jobs/jobs_page_default to perform KV lookups in [3, 5], got 3
    validate_benchmark_data.go:125: success: expected Jobs/resume_job to perform KV lookups in [6, 6], got 6
    validate_benchmark_data.go:125: success: expected ORMQueries/column_descriptions_json_agg to perform KV lookups in [6, 6], got 6
    validate_benchmark_data.go:125: success: expected ORMQueries/liquibase_migrations_on_multiple_dbs to perform KV lookups in [8, 8], got 8
    validate_benchmark_data.go:125: success: expected ORMQueries/prisma_column_descriptions to perform KV lookups in [133, 133], got 133
    validate_benchmark_data.go:125: success: expected ORMQueries/asyncpg_types to perform KV lookups in [0, 0], got 0
    validate_benchmark_data.go:78: see the -rewrite flag to re-run the benchmarks and adjust the expectations
    validate_benchmark_data.go:79: usage: ./dev test --rewrite pkg/bench/rttanalysis -f=TestBenchmarkExpectation/ --test-args '-rewrite-iterations=N'
    validate_benchmark_data.go:112: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/b03b0378e25aebc8fcd368039859e7a0/logTestBenchmarkExpectation2728311517
--- FAIL: TestBenchmarkExpectation (190.64s)
--- FAIL: TestBenchmarkExpectation (191.19s)
=== RUN   TestBenchmarkExpectation/Jobs
    --- FAIL: TestBenchmarkExpectation/Jobs (141.76s)
=== RUN   TestBenchmarkExpectation/Jobs/jobs_page_latest_50
    rtt_analysis_bench.go:215: Jobs/jobs_page_latest_50: got 5, expected 3
    rtt_analysis_bench.go:221: wrote jaeger trace to /artifacts/tmp/_tmp/b03b0378e25aebc8fcd368039859e7a0/logTestBenchmarkExpectation2728311517/Jobs_jobs_page_latest_50.jaeger.json
        --- FAIL: TestBenchmarkExpectation/Jobs/jobs_page_latest_50 (123.84s)
Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)

Same failure on other branches

- #126750 bench/rttanalysis: TestBenchmarkExpectation failed [Jobs/jobs_page_type_filtered_no_matches] [A-jobs C-test-failure O-robot P-2 T-jobs branch-release-24.1]

/cc @cockroachdb/sql-foundations

This test on roachdash | Improve this report!

Jira issue: CRDB-42392

fqazi commented 2 weeks ago

This looks like its linked to the round trip time on a jobs query