pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed

cockroach-teamcity commented 1 week ago

pkg/ccl/testccl/workload/schemachange/schemachange_test.TestWorkload failed with artifacts on release-23.1 @ 873f392bdcb164f25d6f5a38f714187c98e9fde7:

1, 29, rangelog, 13
1, 29, replication_constraint_stats, 25
1, 29, replication_critical_localities, 26
1, 29, replication_stats, 27
1, 29, reports_meta, 28
1, 29, role_id_seq, 48
1, 29, role_members, 23
1, 29, role_options, 33
1, 29, scheduled_jobs, 37
1, 29, settings, 6
1, 29, span_configurations, 47
1, 29, span_stats_buckets, 55
1, 29, span_stats_samples, 56
1, 29, span_stats_tenant_boundaries, 57
1, 29, span_stats_unique_keys, 54
1, 29, sql_instances, 46
1, 29, sqlliveness, 39
1, 29, statement_activity, 60
1, 29, statement_bundle_chunks, 34
1, 29, statement_diagnostics, 36
1, 29, statement_diagnostics_requests, 35
1, 29, statement_statistics, 42
1, 29, table_statistics, 20
1, 29, task_payloads, 58
1, 29, tenant_id_seq, 62
1, 29, tenant_settings, 50
1, 29, tenant_tasks, 59
1, 29, tenant_usage, 45
1, 29, tenants, 8
1, 29, transaction_activity, 61
1, 29, transaction_statistics, 43
1, 29, ui, 14
1, 29, users, 4
1, 29, web_sessions, 19
1, 29, zones, 5
100, 0, public, 101
102, 0, public, 103
104, 0, public, 105
104, 0, schema120, 134
104, 105, _enum118, 133
104, 105, _enum164, 142
104, 105, _enum222, 153
104, 105, enum118, 132
104, 105, enum164, 141
104, 105, enum222, 152
104, 134, seq174, 146
    schema_change_external_test.go:80: backup in /artifacts/tmp/_tmp/fcba020ca7cb44bf8385bd0e13bcbe2d/logTestWorkload547669499
    panic.go:522: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/fcba020ca7cb44bf8385bd0e13bcbe2d/logTestWorkload547669499
--- FAIL: TestWorkload (49.11s)

Parameters:

TAGS=bazel,gss,deadlock
Help

Same failure on other branches

- #129909 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [duplicate key value violates unique constraint] [C-test-failure O-robot P-2 T-sql-foundations branch-master] - #129462 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot P-2 T-sql-foundations branch-release-24.2.1-rc] - #129366 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [duplicate key value violates unique constraint during CREATE UNIQUE INDEX] [C-test-failure O-robot P-2 T-sql-foundations branch-master] - #128953 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed ["enum type does not exist" error code when setting a DEFAULT] [C-test-failure O-robot P-3 T-sql-foundations branch-release-23.2.10-rc]

/cc @cockroachdb/sql-foundations

This test on roachdash | Improve this report!

Jira issue: CRDB-41973

fqazi commented 1 week ago

We ran into the following:

 "ops": [
  "BEGIN",
  {
   "sql": "ALTER SEQUENCE schema120.seq174 RENAME TO schema120.seq225"
  },
  {
   "sql": "CREATE SCHEMA IF NOT EXISTS schema226 AUTHORIZATION root"
  },
  {
   "sql": "CREATE SEQUENCE IF NOT EXISTS schema120.seq225"
  }
 ],
 "expectedExecErrors": "",
 "expectedCommitErrors": "",
 "message": "***UNEXPECTED ERROR; Received an unexpected execution error.: ERROR: unexpected value: raw_bytes:\"k\\211\\030\\026\\001\\244\\002\" timestamp:\u003c\u003e  (SQLSTATE XXUUU)",

Lets decode the raw bytes and try to figure out which statement above first.

cockroach-teamcity commented 1 week ago

pkg/ccl/testccl/workload/schemachange/schemachange_test.TestWorkload failed with artifacts on release-23.1 @ f0c0643e06354e79c8ad1ec6f830c6265f5d339b:

1, 29, lease, 11
1, 29, locations, 21
1, 29, migrations, 40
1, 29, namespace, 30
1, 29, privileges, 51
1, 29, protected_ts_meta, 31
1, 29, protected_ts_records, 32
1, 29, rangelog, 13
1, 29, replication_constraint_stats, 25
1, 29, replication_critical_localities, 26
1, 29, replication_stats, 27
1, 29, reports_meta, 28
1, 29, role_id_seq, 48
1, 29, role_members, 23
1, 29, role_options, 33
1, 29, scheduled_jobs, 37
1, 29, settings, 6
1, 29, span_configurations, 47
1, 29, span_stats_buckets, 55
1, 29, span_stats_samples, 56
1, 29, span_stats_tenant_boundaries, 57
1, 29, span_stats_unique_keys, 54
1, 29, sql_instances, 46
1, 29, sqlliveness, 39
1, 29, statement_activity, 60
1, 29, statement_bundle_chunks, 34
1, 29, statement_diagnostics, 36
1, 29, statement_diagnostics_requests, 35
1, 29, statement_statistics, 42
1, 29, table_statistics, 20
1, 29, task_payloads, 58
1, 29, tenant_id_seq, 62
1, 29, tenant_settings, 50
1, 29, tenant_tasks, 59
1, 29, tenant_usage, 45
1, 29, tenants, 8
1, 29, transaction_activity, 61
1, 29, transaction_statistics, 43
1, 29, ui, 14
1, 29, users, 4
1, 29, web_sessions, 19
1, 29, zones, 5
100, 0, public, 101
102, 0, public, 103
104, 0, public, 105
104, 0, schema31, 114
    schema_change_external_test.go:80: backup in /artifacts/tmp/_tmp/fcba020ca7cb44bf8385bd0e13bcbe2d/logTestWorkload727908131
    panic.go:522: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/fcba020ca7cb44bf8385bd0e13bcbe2d/logTestWorkload727908131
--- FAIL: TestWorkload (7.37s)

Parameters:

TAGS=bazel,gss
Help

Same failure on other branches

- #130501 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot T-sql-foundations branch-master release-blocker] - #129462 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot P-2 T-sql-foundations branch-release-24.2.1-rc] - #128953 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed ["enum type does not exist" error code when setting a DEFAULT] [C-test-failure O-robot P-3 T-sql-foundations branch-release-23.2.10-rc]

This test on roachdash | Improve this report!

rafiss commented 2 days ago

The latest failure is:

{
 "workerId": 1,
 "clientTimestamp": "09:51:42.932895",
 "ops": [
  "BEGIN",
  {
   "sql": "CREATE TABLE schema31.table35 (\"col%v35_36\" VARCHAR NOT NULL, \".col35_37\" DECIMAL NOT NULL, \" col'35_38\" TSVECTOR NOT NULL, \"col3%v5_39\" TIMETZ NOT NULL, col35_40 REGROLE, cðŸ˜«ol35_41 BOOL NOT NULL, col35_42 TIMETZ[], \"col35\\\\xef_43\" TSVECTOR, \"coL35_44\" STRING AS (CASE WHEN col35_40 IS NULL THEN e'^[\\'Ky1v\\x05\\x05':::STRING ELSE e'@f\\x1cQ\\nU':::STRING END) STORED, col35_45 STRING AS (CASE WHEN col35_40 IS NULL THEN e'WaI8K\"\\x07a':::STRING ELSE '\"':::STRING END) STORED, col35_46 STRING NOT NULL AS (lower(\"col%v35_36\")) VIRTUAL, col35_47 STRING AS (CASE WHEN col35_40 IS NULL THEN '':::STRING ELSE e'\\x04n^QZ\\x0ez*':::STRING END) VIRTUAL, col3Ì±5_48 STRING NOT NULL AS (lower(CAST(\" col'35_38\" AS STRING))) VIRTUAL, coðŸ˜›l35_49 STRING NOT NULL AS (lower(\"col%v35_36\")) STORED, INVERTED INDEX (col35_47 gin_trgm_ops ASC), INDEX (cðŸ˜«ol35_41 DESC, coðŸ˜›l35_49, \"col3%v5_39\", \"col%v35_36\" DESC, col35_46, \".col35_37\"), INDEX (col35_45 DESC), FAMILY (\"coL35_44\"), FAMILY (\"col3%v5_39\"), FAMILY (col35_42, cðŸ˜«ol35_41, \" col'35_38\"), FAMILY (coðŸ˜›l35_49, \".col35_37\", col35_45, \"col35\\\\xef_43\", col35_40), FAMILY (\"col%v35_36\"))",
   "potentialExecErr": "0A000,42601"
  },
  {
   "sql": "SELECT t0.\"col3%v5_39\" AS col0,t0.\"coL35_44\" AS col1,t0.\"col3%v5_39\" AS col2,t0.coðŸ˜›l35_49 AS col3,t0.col35_45 AS col4,t0.col35_46 AS col5,t0.\"col%v35_36\" AS col6 FROM schema31.table35 AS t0  FETCH FIRST 1 ROWS ONLY",
   "potentialExecErr": "53100,53200,57014"
  }
 ],
 "expectedExecErrors": "",
 "expectedCommitErrors": "",
 "message": "***UNEXPECTED ERROR; Received an unexpected execution error.: ERROR: relation \"schema31.table35\" does not exist (SQLSTATE 42P01)",
 "errorState": {
  "potentialErrors": [
   "53100",
   "53200",
   "57014"
  ],

It looks like the SELECT right after the CREATE was unable to resolve the table that was just created. In 9f31ac5d3779fdc4dc12a8b0205596793e206aac we disallowed mixed DML and DDL transactions, but that hasn't been backported to this branch.

Let's focus on the other failure with the unexpected key.

cockroachdb / cockroach

pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed #130283