cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30k stars 3.79k forks source link

roachtest: schemachange/random-load failed #70016

Closed cockroach-teamcity closed 2 years ago

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 99a4816fc272228a63df20dae3cc41d235e705f3:

          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:50:15.270454",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA IF NOT EXISTS schema30 AUTHORIZATION root",
          |   "ALTER DATABASE schemachange ADD REGION \"us-west1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot add region \"us-west1\" to database schemachange (SQLSTATE 42P12)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:50:15.685587",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange ADD REGION \"us-east1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot add region \"us-east1\" to database schemachange (SQLSTATE 42P12)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:50:15.267228",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "DROP SCHEMA \"public\" CASCADE"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot drop schema \"public\" (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:50:15.266796",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-west1\"",
          |   "ALTER DATABASE schemachange ADD REGION \"us-west1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: region \"us-west1\" already added to database (SQLSTATE 42710)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

Jira issue: CRDB-9901

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 4811bb9c8c1d20b9f200d44bc7df388ad2d80898:

          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:40:37.375216",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE VIEW public.view87 AS SELECT public.table48.col48_53, public.table48.col48_52, public.table48.col48_49, public.table48.col48_50, public.table48.col48_51, public.table48.col48_52, public.table48.col48_53, public.table48.col48_50, public.table48.col48_49, public.table48.col48_52, public.table48.col48_51 FROM public.table48, public.table48, public.table48"
          |  ],
          |  "expectedExecErrors": "42701,42712",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: source name \"table48\" specified more than once (missing AS clause) (SQLSTATE 42712)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:40:37.860643",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE INDEX index70_82 ON public.table70 (col70_71 DESC) STORING (col70_74)",
          |   "CREATE INDEX IF NOT EXISTS index48_90 ON public.table48 (col48_51, col48_50 DESC, col48_53 DESC, col48_52 ASC)"
          |  ],
          |  "expectedExecErrors": "0A000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: unimplemented: column col48_51 is of type geometry and thus is not indexable (SQLSTATE 0A000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:40:37.546193",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table48 ALTER COLUMN \"col48_52\" DROP DEFAULT",
          |   "CREATE VIEW public.view91 AS SELECT public.table48.col48_51, public.table48.col48_49, public.table48.col48_50, public.table48.col48_53, public.table48.col48_52, public.table48.col48_49, public.table48.col48_51, public.table48.col48_50, public.table48.col48_53, public.table48.col48_50, public.table48.col48_51, public.table48.col48_52, public.table48.col48_49 FROM public.table48, public.table48, public.table48"
          |  ],
          |  "expectedExecErrors": "42701,42712",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: source name \"table48\" specified more than once (missing AS clause) (SQLSTATE 42712)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:40:37.401169",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table48 ALTER COLUMN \"col48_50\" SET NOT NULL",
          |   "ALTER TABLE public.table48 ALTER COLUMN \"col48_52\" DROP NOT NULL"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: Validation FAIL:\nid 68, db schemachange, schema public, name table48: mutation job 692355197303750657: job not found"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 8d491ced731e13da2377cda9961f577d7487d6a0:

          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER TABLE public.table93 ALTER COLUMN IrrelevantColumnName SET DEFAULT \"IrrelevantValue\""
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.table93\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:31:01.703025",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=20ed4685 key=/Table/SystemConfigSpan/Start pri=0.00808979 epo=0 ts=1631532661.916913896,1 min=1631532661.702960463,0 seq=39} lock=true stat=PENDING rts=1631532661.702960463,0 wto=false gul=1631532662.202960463,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:31:01.736907",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA schema63 AUTHORIZATION root",
          |   "CREATE TYPE public.enum94 AS ENUM ('adfd', 'daf')",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=2045f510 key=/Table/SystemConfigSpan/Start pri=0.00055577 epo=0 ts=1631532662.010066433,1 min=1631532661.736822886,0 seq=24} lock=true stat=PENDING rts=1631532661.736822886,0 wto=false gul=1631532662.236822886,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:31:01.653537",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"europe-west2\"",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq95",
          |   "CREATE SCHEMA schema96 AUTHORIZATION root",
          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE"
          |  ],
          |  "expectedExecErrors": "42602",
          |  "expectedCommitErrors": "",
          |  "message": "***FAIL; Failed to receive an execution error when errors were expected"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 14411b999aae710ca0f4a6376d58e302b197281b:

          |  "workerId": 0,
          |  "clientTimestamp": "12:31:38.549179",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TYPE schema4.enum92 AS ENUM ('saa', 's')",
          |   "ALTER DATABASE schemachange SURVIVE REGION FAILURE"
          |  ],
          |  "expectedExecErrors": "22023,42602",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: database must have associated regions before a survival goal can be set (SQLSTATE 42602)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:31:38.17109",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER DATABASE schemachange ADD REGION \"us-east1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot add region \"us-east1\" to database schemachange (SQLSTATE 42P12)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:31:38.17901",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"europe-west2\"",
          |   "DROP SCHEMA \"public\" CASCADE"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot drop schema \"public\" (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:31:38.188685",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE IF NOT EXISTS public.table42 (col42_43 NAME NOT NULL, col42_44 INT4 NULL, col42_45 REGROLE, col42_46 STRING NOT NULL, col42_47 GEOGRAPHY NOT NULL, col42_48 BYTES, col42_49 TIMETZ NULL, col42_50 DECIMAL NULL, col42_51 REGPROC NOT NULL, col42_52 TIMESTAMP[] NOT NULL, col42_53 DECIMAL NULL AS (col42_50 + (-3.422671353499638127E+30):::DECIMAL) VIRTUAL, FAMILY (col42_48), FAMILY (col42_49), FAMILY (col42_43, col42_51), FAMILY (col42_44, col42_46, col42_50, col42_45, col42_47, col42_52))",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED COMMIT ERROR; Received an unexpected commit error: ERROR: relation \"table42\" (71): invalid locality config: database schemachange is multi-region enabled, but table table42 has no locality set (SQLSTATE 42P16)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ d3fc366bcba04ab19f4cb59844212e908e2a9aa8:

          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot create \"schema59.seq60\" because the target database or schema does not exist (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:23:49.157987",
          |  "ops": [
          |   "BEGIN",
          |   "DROP SEQUENCE public.seq66"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.seq66\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:23:48.939175",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER TABLE schema64.table65 ALTER COLUMN IrrelevantColumnName SET NOT NULL"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema64.table65\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:23:49.254526",
          |  "ops": [
          |   "BEGIN",
          |   "DROP SCHEMA \"public\" CASCADE"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot drop schema \"public\" (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:23:48.961179",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-west1\"",
          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE"
          |  ],
          |  "expectedExecErrors": "42602",
          |  "expectedCommitErrors": "",
          |  "message": "***FAIL; Failed to receive an execution error when errors were expected"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ ed81878bb4697c3a0a09e69b8a5053566297b142:

          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: index \"index59_193\" does not exist (SQLSTATE 42704)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:51:04.539385",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE schema179.table180 RENAME TO public.table187"
          |  ],
          |  "expectedExecErrors": "42602,42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema179.table180\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:51:04.605099",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA IF NOT EXISTS schema183 AUTHORIZATION root",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: cannot publish new versions for descriptors: [{schemachange 52 2}], old versions still in use (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:51:04.380373",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table59 ALTER COLUMN \"col59_60\" DROP STORED"
          |  ],
          |  "expectedExecErrors": "42611",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: column \"col59_60\" is not a computed column (SQLSTATE 42611)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:51:03.866095",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table59 RENAME COLUMN \"col59_60\" TO \"col59_182\"",
          |   "CREATE INDEX IF NOT EXISTS index59_199 ON public.table59 (col59_60)"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: column \"col59_60\" does not exist (SQLSTATE 42703)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 0babf97f52ed9e44036851b2a9868e17eeee95ed:

          |   "CREATE SCHEMA IF NOT EXISTS schema63 AUTHORIZATION root",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=d2f2fd93 key=/Table/SystemConfigSpan/Start pri=0.01941626 epo=0 ts=1631881643.409447523,1 min=1631881642.896642076,0 seq=30} lock=true stat=PENDING rts=1631881642.896642076,0 wto=false gul=1631881643.396642076,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:27:22.898491",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=0a47e379 key=/Table/SystemConfigSpan/Start pri=0.01247013 epo=0 ts=1631881643.409447523,1 min=1631881642.898424145,0 seq=24} lock=true stat=PENDING rts=1631881642.898424145,0 wto=false gul=1631881643.398424145,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:27:22.897299",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "CREATE TYPE public.enum64 AS ENUM ('f')",
          |   "CREATE TABLE public.table65 (col65_66 TIMESTAMP NOT NULL, col65_67 CHAR NOT NULL, col65_68 INT8 NOT NULL, PRIMARY KEY (col65_67 DESC, col65_68, col65_66 DESC), col65_69 STRING NOT NULL AS (lower(CAST(col65_66 AS STRING))) STORED, col65_70 STRING NOT NULL AS (lower(CAST(col65_66 AS STRING))) VIRTUAL, UNIQUE (col65_67, lower(CAST(col65_66 AS STRING)) DESC), INDEX (col65_70 ASC, col65_69 ASC), FAMILY (col65_68, col65_69), FAMILY (col65_66, col65_67))",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=82ec5025 key=/Table/SystemConfigSpan/Start pri=0.00227815 epo=0 ts=1631881643.409447523,2 min=1631881642.897230899,0 seq=41} lock=true stat=PENDING rts=1631881642.897230899,0 wto=false gul=1631881643.397230899,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:27:22.898298",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "CREATE TABLE public.table35 (col35_36 VARCHAR NOT NULL, col35_37 TIMESTAMPTZ NULL, col35_38 VARCHAR, col35_39 JSONB NOT NULL, col35_40 INT2 NULL, col35_41 REGROLE NOT NULL, PRIMARY KEY (col35_41 DESC))",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\""
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: Validation FAIL:\nid 63, db schemachange, schema public, name table35: relation \"table35\" (63): invalid locality config: database schemachange is multi-region enabled, but table table35 has no locality set"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ a29fd0ad538d51050d82e92583a8206983622f6b:

          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.seq38\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:55:04.169059",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange SURVIVE REGION FAILURE"
          |  ],
          |  "expectedExecErrors": "22023,42602",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: database must have associated regions before a survival goal can be set (SQLSTATE 42602)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:55:04.174372",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER DATABASE schemachange ADD REGION \"europe-west2\"",
          |   "CREATE SCHEMA IF NOT EXISTS schema41 AUTHORIZATION root",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=d355e271 key=/Table/SystemConfigSpan/Start pri=0.00744406 epo=0 ts=1631966104.484505951,1 min=1631966104.174263983,0 seq=52} lock=true stat=PENDING rts=1631966104.174263983,0 wto=false gul=1631966104.674263983,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:55:04.173522",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA IF NOT EXISTS schema13 AUTHORIZATION root",
          |   "CREATE TABLE public.table42 (col42_43 BOX2D NOT NULL, col42_44 BOOL NOT NULL, col42_45 INET NOT NULL, col42_46 INT4 NOT NULL, col42_47 DECIMAL NOT NULL, col42_48 REGCLASS NOT NULL, col42_49 FLOAT4 NOT NULL, PRIMARY KEY (col42_45 ASC, col42_46, col42_48 DESC, col42_43 DESC, col42_49 ASC, col42_47, col42_44 ASC), col42_50 STRING NOT NULL AS (lower(CAST(col42_48 AS STRING))) STORED, col42_51 DECIMAL NOT NULL AS (col42_47 + 1570938224636458.367:::DECIMAL) STORED, UNIQUE (col42_43 ASC, col42_50 ASC, col42_47 ASC) STORING (col42_51))",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq52",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-west1\"",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED COMMIT ERROR; Received an unexpected commit error: ERROR: relation \"table42\" (73): invalid locality config: database schemachange is multi-region enabled, but table table42 has no locality set (SQLSTATE 42P16)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 5202c97fcfe7e51e244335c3cd68bd33647860ff:

          |  "workerId": 0,
          |  "clientTimestamp": "12:05:17.955325",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange ADD REGION \"us-west1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot add region \"us-west1\" to database schemachange (SQLSTATE 42P12)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:05:17.890137",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "DROP SEQUENCE IF EXISTS public.seq137",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": ""
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:05:17.873511",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SEQUENCE IF NOT EXISTS schema1.seq133",
          |   "ALTER TABLE public.table152 ALTER COLUMN \"IrrelevantColumnName\" DROP DEFAULT"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.table152\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:05:17.376042",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA schema94 AUTHORIZATION root",
          |   "CREATE TABLE schema1.table120 (col120_121 TIME NOT NULL, col120_122 NAME NOT NULL, col120_123 BYTES NOT NULL, col120_124 DECIMAL NULL, col120_125 VARCHAR NOT NULL, col120_126 REGPROC NULL, col120_127 TIME NOT NULL, col120_128 TIMESTAMP NOT NULL, col120_129 INT4 NOT NULL, col120_130 STRING, PRIMARY KEY (col120_128, col120_125, col120_127, col120_129 DESC, col120_123), col120_131 STRING NULL AS (lower(CAST(col120_126 AS STRING))) VIRTUAL, INDEX ((CASE WHEN col120_123 IS NULL THEN e'=\\x03aW2\u003eq ':::STRING ELSE 'X':::STRING END) DESC) STORING (col120_121, col120_124, col120_126), INDEX (col120_131, col120_122 DESC, col120_124 ASC, (col120_124 + (-1.234E+401):::DECIMAL) DESC, col120_125 ASC, col120_127 ASC, col120_126, lower(col120_122) ASC, col120_129 ASC, col120_128 DESC, col120_123 DESC) STORING (col120_130), INDEX (col120_127 ASC, col120_121 DESC, col120_126 ASC, col120_125 ASC, col120_123, col120_124 ASC), UNIQUE (col120_122 DESC, col120_121 DESC, col120_131, col120_127, col120_126), UNIQUE (col120_131 DESC, col120_122 ASC, col120_123, col120_125 ASC, col120_124 ASC, col120_130, lower(col120_122), col120_129 ASC, col120_121 ASC, col120_128 DESC, col120_126), INDEX (col120_128), INDEX (col120_124 ASC, col120_128 DESC, col120_123, col120_130 DESC, col120_122 DESC) STORING (col120_121), FAMILY (col120_127), FAMILY (col120_124, col120_130, col120_123, col120_128, col120_122), FAMILY (col120_125, col120_126, col120_129, col120_121))",
          |   "CREATE INDEX index120_136 ON schema1.table120 (col120_125, col120_126 DESC, col120_124 ASC, crdb_internal_idx_expr_3 DESC, col120_121 ASC, col120_123, crdb_internal_idx_expr DESC, col120_130 DESC, col120_128 ASC, col120_129, crdb_internal_idx_expr_2 DESC, col120_122 DESC, col120_127) STORING (crdb_internal_idx_expr_1)"
          |  ],
          |  "expectedExecErrors": "XXUUU",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: column \"crdb_internal_idx_expr_3\" is inaccessible and cannot be referenced (SQLSTATE 42703)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ db7de996a04d0b7e7959decd06c2cd40d80e501f:

          |  "workerId": 0,
          |  "clientTimestamp": "12:07:23.448217",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"europe-west2\"",
          |   "CREATE TABLE schema7.table203 (col203_204 BYTES NOT NULL, col203_205 CHAR NOT NULL, col203_206 BIT(45) NOT NULL, PRIMARY KEY (col203_204, col203_206 ASC, col203_205 ASC), col203_207 STRING NOT NULL AS (CASE WHEN col203_204 IS NULL THEN 'n]':::STRING ELSE '\"':::STRING END) VIRTUAL, col203_208 STRING NOT NULL AS (lower(col203_205)) VIRTUAL, INDEX (col203_204), UNIQUE (col203_206 ASC, col203_205, col203_204), UNIQUE (col203_206 DESC, col203_204 DESC, col203_205, col203_208 DESC), UNIQUE (col203_204 ASC, col203_208 ASC, col203_205))",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=b3316542 key=/Table/SystemConfigSpan/Start pri=0.01983275 epo=0 ts=1632312444.058692195,1 min=1632312443.447963048,0 seq=32} lock=true stat=PENDING rts=1632312443.447963048,0 wto=false gul=1632312443.947963048,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:07:24.059254",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange ADD REGION \"europe-west2\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot add region \"europe-west2\" to database schemachange (SQLSTATE 42P12)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:07:23.349814",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA schema178 AUTHORIZATION root",
          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE"
          |  ],
          |  "expectedExecErrors": "42602",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: database must have associated regions before a survival goal can be set (SQLSTATE 42602)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:07:23.53197",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq174",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-west1\""
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: region \"us-west1\" has not been added to the database (SQLSTATE 42602)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 184cac1592349a665c39a54e688de072388e7886:

          |   "ALTER DATABASE schemachange SURVIVE REGION FAILURE"
          |  ],
          |  "expectedExecErrors": "22023,42602",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: at least 3 regions are required for surviving a region failure (SQLSTATE 22023)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:59:55.816312",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq101",
          |   "DROP SEQUENCE public.seq52",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=a88b7b49 key=/Table/SystemConfigSpan/Start pri=0.00242121 epo=0 ts=1632398396.230254311,1 min=1632398395.816239182,0 seq=21} lock=true stat=PENDING rts=1632398395.816239182,0 wto=false gul=1632398396.316239182,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:59:55.805055",
          |  "ops": [
          |   "BEGIN",
          |   "DROP SEQUENCE IF EXISTS public.seq52",
          |   "CREATE SCHEMA schema119 AUTHORIZATION root",
          |   "CREATE TYPE public.enum120 AS ENUM ('d', 'f', 'ddsa', 'fadads', 'dsfd')",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq121",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=6af9fb0a key=/Table/SystemConfigSpan/Start pri=0.13024011 epo=0 ts=1632398396.903724155,1 min=1632398395.804993512,0 seq=45} lock=true stat=PENDING rts=1632398395.804993512,0 wto=false gul=1632398396.304993512,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:59:55.848977",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE IF NOT EXISTS public.table103 (col103_104 BIT(22), col103_105 INET NOT NULL, col103_106 INET NULL, col103_107 \"char\", col103_108 REGCLASS NOT NULL, col103_109 FLOAT4 NOT NULL, col103_110 REGNAMESPACE NULL, col103_111 DECIMAL NOT NULL, col103_112 JSONB, col103_113 GEOMETRY NOT NULL, col103_114 BOOL NOT NULL, PRIMARY KEY (col103_105 ASC, col103_108 DESC, col103_111 DESC, col103_109 DESC, col103_114), col103_115 STRING AS (lower(CAST(col103_104 AS STRING))) STORED, col103_116 STRING AS (lower(CAST(col103_112 AS STRING))) STORED, INDEX (lower(CAST(col103_105 AS STRING)) DESC, col103_111, col103_106, col103_104), UNIQUE (col103_114 DESC, col103_104 DESC, lower(CAST(col103_114 AS STRING)), col103_110 ASC), INDEX (col103_116, col103_109 DESC) STORING (col103_106, col103_113), FAMILY (col103_116, col103_109), FAMILY (col103_112), FAMILY (col103_115), FAMILY (col103_114, col103_110), FAMILY (col103_105, col103_104, col103_106, col103_107), FAMILY (col103_113), FAMILY (col103_111), FAMILY (col103_108))",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED COMMIT ERROR; Received an unexpected commit error: ERROR: relation \"table103\" (98): invalid locality config: database schemachange is multi-region enabled, but table table103 has no locality set (SQLSTATE 42P16)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ c46f5a5a098577b936e56f03d20c97300b4cce61:

          |  "workerId": 0,
          |  "clientTimestamp": "11:47:51.228806",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TYPE public.enum2 AS ENUM ('fddf')",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "CREATE INDEX \"IrrelevantName\" ON schema37.table38 (\"IrrelevantColumn\" ASC)"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema37.table38\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:47:51.229228",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange SURVIVE REGION FAILURE"
          |  ],
          |  "expectedExecErrors": "22023,42602",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: database must have associated regions before a survival goal can be set (SQLSTATE 42602)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:47:51.228263",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "CREATE TABLE schema39.table40 (col40_41 TIMESTAMP, col40_42 OID NULL, col40_43 GEOGRAPHY NULL, col40_44 OID, col40_45 GEOGRAPHY, col40_46 OID NOT NULL, col40_47 JSONB NOT NULL, col40_48 CHAR NULL, col40_49 FLOAT8 NOT NULL, col40_50 GEOMETRY NOT NULL, col40_51 REGCLASS NULL, col40_52 REGPROCEDURE NOT NULL, col40_53 INT2 NOT NULL, col40_54 STRING NULL AS (lower(CAST(col40_51 AS STRING))) VIRTUAL, col40_55 STRING NULL AS (lower(CAST(col40_51 AS STRING))) VIRTUAL, col40_56 STRING AS (lower(CAST(col40_41 AS STRING))) STORED, col40_57 STRING NOT NULL AS (lower(CAST(col40_50 AS STRING))) STORED, col40_58 FLOAT8 NOT NULL AS (col40_49 + 'NaN':::FLOAT8) STORED, INVERTED INDEX (col40_47 ASC), FAMILY (col40_41, col40_49), FAMILY (col40_44, col40_48), FAMILY (col40_45), FAMILY (col40_56, col40_46), FAMILY (col40_50, col40_52), FAMILY (col40_43, col40_53), FAMILY (col40_58, col40_42), FAMILY (col40_51), FAMILY (col40_57), FAMILY (col40_47))"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot create \"schema39.table40\" because the target database or schema does not exist (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:47:51.228764",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA schema1 AUTHORIZATION root",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-west1\"",
          |   "ALTER DATABASE schemachange ADD REGION \"europe-west2\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "***FAIL; Failed to receive an execution error when errors were expected"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 24021ba163e4ac438b169d575cf1527a4aae394d:

          |   | ``````
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 --txn-log /mnt/data1/cockroach/transactions.json
          |   | ``````
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | :37:47.601296",
          |  "ops": [
          |   "BEGIN",
          |   "DROP SEQUENCE public.seq173",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "CREATE TABLE IF NOT EXISTS public.table346 (col346_347 GEOGRAPHY NOT NULL, col346_348 REGNAMESPACE NOT NULL, col346_349 GEOMETRY, col346_350 INT8, col346_351 INT4, col346_352 BOX2D, col346_353 INT8 NULL, col346_354 UUID NOT NULL, col346_355 DATE NOT NULL, FAMILY (col346_351, col346_350), FAMILY (col346_352, col346_354, col346_355), FAMILY (col346_347), FAMILY (col346_349, col346_348, col346_353))",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=b66c34e9 key=/Table/SystemConfigSpan/Start pri=0.01001218 epo=0 ts=1632573467.837100539,1 min=1632573467.601209870,0 seq=20} lock=true stat=PENDING rts=1632573467.601209870,0 wto=false gul=1632573468.101209870,0 (SQLSTATE 40001)"
          | }
          |    10.0s        0            7.0           12.8     65.0    251.7    251.7    251.7 opOk
          |    10.0s        0            1.0            1.5   2684.4   2684.4   2684.4   2684.4 txnCmtErr
          |    10.0s        0            0.0            1.2      0.0      0.0      0.0      0.0 txnOk
          |    10.0s        0            4.0           11.9      0.9   2013.3   2013.3   2013.3 txnRbk
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:37:47.505387",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TYPE public.enum228 AS ENUM ('df', 'sddsd', 'da', 'fsfsdd', 'saffds', 'daf')"
          |  ],
          |  "expectedExecErrors": "42710",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: type \"schemachange.public.enum228\" already exists (SQLSTATE 42710)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:37:47.698774",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE IF NOT EXISTS public.table307 (col307_308 UUID NOT NULL, col307_309 GEOGRAPHY, col307_310 INT2 NOT NULL, col307_311 REGTYPE, col307_312 DECIMAL, col307_313 OID, col307_314 OID, col307_315 \"char\" NOT NULL, col307_316 INT2 NULL, col307_317 REGPROCEDURE NOT NULL, col307_318 REGROLE, col307_319 GEOMETRY, col307_320 OID NOT NULL, col307_321 TIMETZ, col307_322 OID NOT NULL, col307_323 INT2 NULL AS (col307_316 + 22778:::INT8) STORED, col307_324 STRING AS (lower(CAST(col307_321 AS STRING))) STORED, col307_325 STRING AS (lower(CAST(col307_311 AS STRING))) VIRTUAL, col307_326 DECIMAL AS (col307_312 + '-Infinity':::DECIMAL) STORED, UNIQUE (col307_322, col307_312, col307_323, col307_318, col307_311 ASC, col307_324, col307_316 DESC, col307_317 DESC, col307_320 ASC, col307_313, col307_326, col307_314, col307_325 DESC, col307_321 DESC) STORING (col307_309, col307_310, col307_315, col307_319), UNIQUE (col307_313, col307_323, col307_322 DESC, col307_317, col307_326 DESC) STORING (col307_308, col307_311, col307_312, col307_316, col307_321, col307_324), UNIQUE (lower(CAST(col307_308 AS STRING)) DESC, col307_313 ASC, col307_323 ASC, col307_320 DESC) STORING (col307_308, col307_311, col307_312, col307_317, col307_318, col307_321, col307_322, col307_324), FAMILY (col307_315), FAMILY (col307_322, col307_311, col307_309, col307_326), FAMILY (col307_308), FAMILY (col307_317), FAMILY (col307_324, col307_314), FAMILY (col307_310), FAMILY (col307_319, col307_316, col307_312, col307_321, col307_313, col307_323), FAMILY (col307_318, col307_320))",
          |   "ALTER TABLE public.table307 ADD COLUMN col307_345 OID NULL",
          |   "CREATE INDEX IF NOT EXISTS index307_356 ON public.table307 (col307_309, col307_323 DESC, col307_319 DESC, col307_326 ASC, col307_314 ASC, col307_312 DESC, col307_316 DESC, col307_318, col307_311, col307_313 ASC, col307_308, crdb_internal_idx_expr, col307_322, col307_317 ASC) STORING (col307_324, col307_325, col307_310, col307_320)"
          |  ],
          |  "expectedExecErrors": "0A000,XXUUU",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: column \"crdb_internal_idx_expr\" is inaccessible and cannot be referenced (SQLSTATE 42703)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 947d24dc88c05bf698d8011e1b5cf3b2017f26a3:

          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE public.table86 AS SELECT \"IrrelevantColumnName\", public.table43.col43_44, public.table43.col43_49, public.table43.col43_54, public.table43.col43_48, public.table43.col43_46, public.table43.col43_45, public.table43.col43_56, public.table43.col43_52, public.table43.crdb_internal_idx_expr_1, public.table43.crdb_internal_idx_expr_3, public.table43.col43_51, public.table43.crdb_internal_idx_expr_4, public.table43.col43_58, public.table43.col43_50, public.table43.col43_47, public.table43.crdb_internal_idx_expr_2, public.table43.col43_53, public.table43.crdb_internal_idx_expr, public.table43.col43_57, public.table43.col43_55 FROM public.table81, public.table43"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.table81\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:06:23.284517",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE schema82.table83 RENAME COLUMN \"IrrelevantColumnName\" TO \"OtherIrrelevantName\""
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema82.table83\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:06:23.332893",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "CREATE SEQUENCE IF NOT EXISTS schema1.seq87",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=331f476b key=/Table/SystemConfigSpan/Start pri=0.00679004 epo=0 ts=1632830783.395552058,1 min=1632830783.332797968,0 seq=8} lock=true stat=PENDING rts=1632830783.332797968,0 wto=false gul=1632830783.832797968,0 (SQLSTATE 40001)"
          | }
          | _elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
          |     1.0s        0           18.4           20.0     12.1    109.1    184.5    184.5 opOk
          |     1.0s        0            1.8            2.0    285.2    402.7    402.7    402.7 txnCmtErr
          |     1.0s        0            0.9            1.0    671.1    671.1    671.1    671.1 txnOk
          |     1.0s        0           30.4           33.0     12.6    201.3    268.4    268.4 txnRbk
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:06:23.395683",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE INDEX index43_88 ON public.table43 (crdb_internal_idx_expr_3 DESC, col43_48, col43_49, col43_51, crdb_internal_idx_expr_4 ASC, col43_52, col43_54 DESC, col43_45, crdb_internal_idx_expr_1 ASC, col43_50 ASC, crdb_internal_idx_expr ASC, col43_53 ASC) STORING (col43_46, crdb_internal_idx_expr_2, col43_47, col43_58, col43_44, col43_56, col43_55, col43_57)"
          |  ],
          |  "expectedExecErrors": "0A000,XXUUU",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: column \"crdb_internal_idx_expr_3\" is inaccessible and cannot be referenced (SQLSTATE 42703)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 8d1d2b688ceedb0e5577d1da789da74d060c92ad:

          |   "CREATE TABLE public.table85 (col85_86 REGTYPE, col85_87 REGCLASS NULL, col85_88 TIME NULL, col85_89 JSONB NULL, col85_90 REGNAMESPACE NULL, col85_91 REGNAMESPACE NOT NULL, col85_92 JSONB, col85_93 GEOGRAPHY NULL, col85_94 STRING NULL AS (lower(CAST(col85_87 AS STRING))) VIRTUAL, col85_95 STRING AS (lower(CAST(col85_92 AS STRING))) STORED, col85_96 STRING NULL AS (lower(CAST(col85_87 AS STRING))) STORED, col85_97 STRING NULL AS (lower(CAST(col85_93 AS STRING))) STORED, col85_98 STRING NULL AS (lower(CAST(col85_90 AS STRING))) VIRTUAL, INVERTED INDEX (col85_89), FAMILY (col85_91, col85_96, col85_88, col85_97), FAMILY (col85_92, col85_90, col85_87), FAMILY (col85_95, col85_93), FAMILY (col85_89), FAMILY (col85_86))",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=120cb015 key=/Table/SystemConfigSpan/Start pri=0.03352310 epo=0 ts=1632916331.921246505,1 min=1632916331.669613803,0 seq=7} lock=true stat=PENDING rts=1632916331.669613803,0 wto=false gul=1632916332.169613803,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:52:11.748089",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq101",
          |   "DROP TABLE schema1.table111"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema1.table111\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:52:11.919429",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SEQUENCE IF NOT EXISTS schema1.seq105",
          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE"
          |  ],
          |  "expectedExecErrors": "42602",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: database must have associated regions before a survival goal can be set (SQLSTATE 42602)"
          | }
          |     2.0s        0            6.0           14.5    939.5   1208.0   1208.0   1208.0 opOk
          |     2.0s        0            2.0            1.5    939.5   1140.9   1140.9   1140.9 txnCmtErr
          |     2.0s        0            0.5            0.5   1208.0   1208.0   1208.0   1208.0 txnOk
          |     2.0s        0            3.0           17.5      0.8      0.8      0.8      0.8 txnRbk
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:52:11.511721",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"europe-west2\""
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: region \"europe-west2\" has not been added to the database (SQLSTATE 42602)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 88d8c1ce72a3988332e502e103421996d3908ab3:

          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.table273\" does not exist (SQLSTATE 42P01)"
          | }
          |     3.0s        0           28.0           30.3     37.7    402.7    436.2    436.2 opOk
          |     3.0s        0            4.0            4.3    570.4    939.5    939.5    939.5 txnCmtErr
          |     3.0s        0            4.0            2.7    604.0   2684.4   2684.4   2684.4 txnOk
          |     3.0s        0           26.0           25.7      5.8    260.0    352.3    352.3 txnRbk
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:57:29.316934",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange ADD REGION \"europe-west2\"",
          |   "ALTER DATABASE schemachange ADD REGION \"europe-west2\""
          |  ],
          |  "expectedExecErrors": "42710",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: region \"europe-west2\" already added to database (SQLSTATE 42710)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:57:29.183019",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA schema242 AUTHORIZATION root",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=34551422 key=/Table/SystemConfigSpan/Start pri=0.00889963 epo=0 ts=1633003049.844152512,1 min=1633003049.182927886,0 seq=31} lock=true stat=PENDING rts=1633003049.182927886,0 wto=false gul=1633003049.682927886,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:57:29.452395",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE IF NOT EXISTS public.table243 (col243_244 TIMESTAMP NULL, col243_245 NAME, col243_246 DECIMAL NULL, col243_247 REGTYPE NOT NULL, col243_248 DATE, INDEX (col243_244 ASC, col243_245 ASC, col243_247, col243_246, col243_248 ASC), UNIQUE (col243_244 DESC, col243_245 DESC, col243_247 ASC), UNIQUE (col243_246, col243_245 DESC), INDEX (col243_245, col243_246, lower(col243_245) ASC, col243_247, col243_248 DESC), INDEX (col243_247, col243_244 DESC, col243_245, col243_246 DESC, col243_248 DESC), UNIQUE (col243_244, col243_247 DESC) STORING (col243_245, col243_246, col243_248))",
          |   "ALTER TABLE public.table243 RENAME COLUMN \"col243_247\" TO \"col243_276\"",
          |   "DROP TABLE public.table243 RESTRICT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: ERROR: relation \"public.table243\" does not exist (SQLSTATE 42P01)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 57bb9046cd2a3090dc5f17b79912f34db8dde33e:

          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | SC, col90_95, col90_109 DESC, col90_105 ASC, col90_100 ASC, col90_103 ASC, col90_107 DESC, col90_93 ASC), UNIQUE (col90_95, col90_106, col90_102 DESC, col90_103, col90_93, col90_91 ASC, col90_104, col90_108 ASC, col90_100, col90_98 DESC, col90_94 ASC, col90_109), INDEX (col90_108 DESC, col90_93 ASC, lower(CAST(col90_91 AS STRING)), col90_107) STORING (col90_91, col90_92, col90_95, col90_100), INDEX (col90_105, col90_103 DESC) STORING (col90_91, col90_92, col90_93, col90_95, col90_100, col90_102, col90_108))",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: cannot publish new versions for descriptors: [{schemachange 52 1}], old versions still in use (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:51:12.889604",
          |  "ops": [
          |   "BEGIN",
          |   "DROP SCHEMA \"schema76\" CASCADE"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: unknown schema \"schema76\" (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:51:12.368821",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA IF NOT EXISTS schema69 AUTHORIZATION root",
          |   "CREATE TABLE schema1.table113 (col113_114 INTERVAL, col113_115 INT2 NULL, col113_116 REGTYPE, col113_117 DECIMAL NULL, col113_118 BIT(48) NOT NULL, col113_119 OID NULL, col113_120 TIMETZ, col113_121 OID NULL, col113_122 INT8 NULL, col113_123 INT8 NOT NULL, col113_124 TIME, col113_125 INT2 NULL AS (col113_115 + 2021:::INT8) STORED, col113_126 INT8 NULL AS (col113_122 + NULL) VIRTUAL, col113_127 INT8 NULL AS (col113_122 + 6742261453571753526:::INT8) VIRTUAL, col113_128 INT2 NULL AS (col113_115 + col113_123) VIRTUAL, UNIQUE (col113_115 DESC, col113_116 ASC, col113_126 DESC, col113_124 ASC, col113_121, ((col113_122 + col113_115) + col113_123) ASC, col113_125, lower(CAST(col113_120 AS STRING)), col113_118), UNIQUE (col113_127 ASC, col113_120, (col113_115 + (-18802):::INT8)), UNIQUE (col113_114, col113_115 ASC, col113_122 ASC, col113_119, col113_126 DESC, col113_124 DESC, col113_121 ASC, col113_118 DESC, col113_123 DESC, col113_125, col113_120 DESC, col113_116 ASC) STORING (col113_117), INDEX (col113_126, col113_114 DESC, col113_128 ASC, col113_117, col113_116) STORING (col113_115, col113_118, col113_119, col113_121, col113_123), UNIQUE (col113_122 ASC, col113_124 DESC, col113_115 ASC, col113_125 ASC, col113_123 DESC, col113_116, col113_126 DESC, col113_121 ASC), INDEX (col113_121 ASC, col113_123 DESC, col113_128 DESC, col113_118 ASC, col113_122 ASC, col113_124, col113_117 ASC, col113_119 DESC, col113_126 DESC, col113_116 ASC) STORING (col113_115), INDEX (col113_122 DESC, col113_117 ASC, col113_125, col113_118 ASC, col113_124 ASC, col113_115, col113_127 ASC, col113_114 ASC, col113_120 DESC, col113_119 ASC, col113_116 DESC, col113_126 ASC, col113_121, col113_123, col113_128 ASC), FAMILY (col113_124), FAMILY (col113_123, col113_117, col113_115), FAMILY (col113_119, col113_125), FAMILY (col113_116), FAMILY (col113_121, col113_118, col113_114), FAMILY (col113_120), FAMILY (col113_122))",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=6745ce3c key=/Table/SystemConfigSpan/Start pri=0.01190756 epo=0 ts=1633089073.042178737,1 min=1633089072.368770643,0 seq=22} lock=true stat=PENDING rts=1633089072.368770643,0 wto=false gul=1633089072.868770643,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:51:12.720434",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"europe-west2\"",
          |   "ALTER DATABASE schemachange ADD REGION \"us-west1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "***FAIL; Failed to receive an execution error when errors were expected"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 47b426a9d692a9faebb3749f25fcbfef4f64af34:

          |   "CREATE TYPE public.enum198 AS ENUM ('daa', 'ass', 'fd', 'ffa')",
          |   "CREATE TYPE public.enum214 AS ENUM ('faafd', 'sda')",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER TABLE public.table218 ALTER COLUMN IrrelevantColumnName DROP STORED"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.table218\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:05:35.164361",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA schema197 AUTHORIZATION root",
          |   "DROP SCHEMA \"public\" CASCADE"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot drop schema \"public\" (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:05:35.174714",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE public.table199 (col199_200 INET NOT NULL, col199_201 INT4 NOT NULL, PRIMARY KEY (col199_201 DESC, col199_200 ASC))",
          |   "DROP TABLE IF EXISTS public.table199 RESTRICT",
          |   "ALTER DATABASE schemachange ADD REGION \"us-east1\"",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=b1214350 key=/Table/SystemConfigSpan/Start pri=0.00275676 epo=0 ts=1633176335.791423360,1 min=1633176335.174654868,0 seq=35} lock=true stat=PENDING rts=1633176335.174654868,0 wto=false gul=1633176335.674654868,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:05:34.705487",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE IF NOT EXISTS public.table185 (col185_186 TIMESTAMPTZ NOT NULL, col185_187 BOOL, col185_188 INT4 NOT NULL, col185_189 BYTES NOT NULL, col185_190 BOOL NOT NULL, col185_191 INTERVAL NOT NULL, col185_192 CHAR[] NOT NULL, PRIMARY KEY (col185_189 DESC, col185_191 ASC, col185_190 DESC), FAMILY (col185_186), FAMILY (col185_187, col185_189, col185_190, col185_191), FAMILY (col185_192, col185_188))",
          |   "CREATE TABLE public.table216 AS SELECT public.table185.col185_186, public.table185.col185_192, public.table185.col185_187, public.table185.col185_188 FROM public.table185",
          |   "ALTER TABLE public.table185 RENAME TO public.table219"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: scanBool: \"\\n\\t\\tSELECT json_array_length(\\n        crdb_internal.pb_to_json(\\n            'cockroach.sql.sqlbase.Descriptor',\\n            descriptor\\n        )-\u003e'table'-\u003e'mutations'\\n       )\\n       \u003e 0\\n\\t\\tFROM system.descriptor\\n\\t  WHERE id = $1::REGCLASS\\n\\t\\t\" [\"public.table185\"]: ERROR: error in argument for $1: relation \"public.table185\" does not exist (SQLSTATE 42P01)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ d1231cff60125b397ccce6c79c9aeea771cdcca4:

          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"europe-west2\"",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=fa327ffa key=/Table/SystemConfigSpan/Start pri=0.00875900 epo=0 ts=1633262270.756410651,1 min=1633262270.157623079,0 seq=24} lock=true stat=PENDING rts=1633262270.157623079,0 wto=false gul=1633262270.657623079,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:57:50.172783",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=2f2ff736 key=/Table/SystemConfigSpan/Start pri=0.01732316 epo=0 ts=1633262270.759287899,1 min=1633262270.172700288,0 seq=24} lock=true stat=PENDING rts=1633262270.172700288,0 wto=false gul=1633262270.672700288,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:57:50.125448",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA IF NOT EXISTS schema60 AUTHORIZATION root",
          |   "CREATE SCHEMA IF NOT EXISTS schema152 AUTHORIZATION root",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=fc1d1195 key=/Table/SystemConfigSpan/Start pri=0.00509443 epo=0 ts=1633262270.759287899,1 min=1633262270.125367507,0 seq=15} lock=true stat=PENDING rts=1633262270.125367507,0 wto=false gul=1633262270.625367507,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:57:50.095822",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"europe-west2\"",
          |   "CREATE SCHEMA IF NOT EXISTS schema207 AUTHORIZATION root",
          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE"
          |  ],
          |  "expectedExecErrors": "42602",
          |  "expectedCommitErrors": "",
          |  "message": "***FAIL; Failed to receive an execution error when errors were expected"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ bd00c351494b44b31178350e79c72244c93ea224:

          |   "BEGIN",
          |   "CREATE TYPE public.enum58 AS ENUM ('fsaada', 'fa', 'dfaaa', 'fda', 'affd', 'fsf')",
          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE"
          |  ],
          |  "expectedExecErrors": "42602",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: database must have associated regions before a survival goal can be set (SQLSTATE 42602)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:39:56.380378",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER TABLE public.table60 ALTER COLUMN \"IrrelevantColumnName\" DROP NOT NULL"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.table60\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:39:56.381109",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "ALTER DATABASE schemachange ADD REGION \"us-west1\"",
          |   "ALTER TABLE public.table61 ALTER COLUMN IrrelevantColumnName SET NOT NULL"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.table61\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:39:56.391862",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq20",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-west1\"",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq62"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: Validation FAIL:\nid 56, db schemachange, schema public, name seq20: relation \"seq20\" (56): invalid locality config: database schemachange is multi-region enabled, but table seq20 has no locality set"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 95c3da44a6c38aa1ae22fe9de5549632b7527aa5:

          |  "ops": [
          |   "BEGIN",
          |   "CREATE TYPE public.enum133 AS ENUM ('asddaf', 'sasf', 'sfad')",
          |   "ALTER TABLE public.table157 ALTER COLUMN IrrelevantColumnName DROP STORED"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.table157\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:19:18.739307",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SEQUENCE IF NOT EXISTS schema25.seq156",
          |   "ALTER TABLE schema158.table159 SET LOCALITY REGIONAL BY ROW"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema158.table159\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:19:18.741512",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TYPE schema25.enum136 AS ENUM ('saf')",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=057b5b67 key=/Table/SystemConfigSpan/Start pri=0.00749929 epo=0 ts=1633436358.783863014,1 min=1633436358.741463884,0 seq=9} lock=true stat=PENDING rts=1633436358.741463884,0 wto=false gul=1633436359.241463884,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:19:18.59958",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA IF NOT EXISTS schema86 AUTHORIZATION root",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-west1\"",
          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE"
          |  ],
          |  "expectedExecErrors": "42602",
          |  "expectedCommitErrors": "",
          |  "message": "***FAIL; Failed to receive an execution error when errors were expected"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ f542fd2731127dab80599943b6041364fcc920e5:

          |  "ops": [
          |   "BEGIN",
          |   "CREATE TYPE public.enum67 AS ENUM ('fd', 'dffd', 'dfs')",
          |   "ALTER TABLE public.table38 ALTER COLUMN \"col38_53\" DROP STORED",
          |   "DROP SEQUENCE public.seq54",
          |   "CREATE INDEX \"IrrelevantName\" ON public.table104 (\"IrrelevantColumn\" ASC)"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.table104\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:30:01.159904",
          |  "ops": [
          |   "BEGIN",
          |   "DROP SCHEMA \"public\" CASCADE"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot drop schema \"public\" (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:30:01.10405",
          |  "ops": [
          |   "BEGIN",
          |   "DROP INDEX public.table90@\"IrrelevantIndexName\""
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.table90\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:30:00.406373",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table38 ALTER COLUMN \"col38_41\" DROP DEFAULT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): \"sql txn\" meta={id=1d7a505a key=/Table/SystemConfigSpan/Start pri=0.00387994 epo=0 ts=1633523401.618042944,1 min=1633523400.406317713,0 seq=1} lock=true stat=PENDING rts=1633523400.406317713,0 wto=false gul=1633523400.906317713,0 (SQLSTATE 40001)"
          | }
          |     2.0s        0           18.2           24.5      7.1    570.4    671.1    671.1 opOk
          |     2.0s        0            8.1            4.5   1275.1   1543.5   1543.5   1543.5 txnCmtErr
          |     2.0s        0            0.0            1.0      0.0      0.0      0.0      0.0 txnOk
          |     2.0s        0           10.1           19.0     23.1    285.2    285.2    285.2 txnRbk
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 82e8782453b1cf14460d93b6bf8328a7b2964575:

          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=aa6024d4 key=/Table/SystemConfigSpan/Start pri=0.03057793 epo=0 ts=1633607190.949621491,2 min=1633607190.411337023,0 seq=32} lock=true stat=PENDING rts=1633607190.411337023,0 wto=false gul=1633607190.911337023,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:46:30.336363",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): \"sql txn\" meta={id=c0000a3d key=/Table/SystemConfigSpan/Start pri=0.01499862 epo=0 ts=1633607191.267678248,1 min=1633607190.336312593,0 seq=8} lock=true stat=PENDING rts=1633607190.336312593,0 wto=false gul=1633607190.836312593,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:46:30.651781",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE VIEW public.view219 AS SELECT public.table180.col180_181, public.table180.col180_187, public.table180.col180_193, public.table180.col180_186, public.table180.col180_195, public.table180.col180_190, public.table180.col180_189, public.table180.col180_191, public.table180.col180_183, public.table180.col180_193, public.table180.col180_194, public.table180.col180_192, public.table180.col180_189, public.table197.col197_201, public.table197.col197_205, public.table197.col197_206, public.table197.col197_204, public.table197.col197_203, public.table197.crdb_internal_idx_expr_1, public.table197.col197_199 FROM public.table180, public.table180, public.table197"
          |  ],
          |  "expectedExecErrors": "42701,42712",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: source name \"table180\" specified more than once (missing AS clause) (SQLSTATE 42712)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:46:30.599537",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table180 ALTER COLUMN \"col180_194\" DROP NOT NULL",
          |   "ALTER TABLE public.table180 ALTER COLUMN \"col180_217\" DROP DEFAULT"
          |  ],
          |  "expectedExecErrors": "42703",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: column \"col180_217\" does not exist (SQLSTATE 42703)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:46:30.738991",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table180 ALTER COLUMN col180_194 SET DEFAULT 219002036:::INT8"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: scanBool: \"\\n\\t\\tSELECT json_array_length(\\n        crdb_internal.pb_to_json(\\n            'cockroach.sql.sqlbase.Descriptor',\\n            descriptor\\n        )-\u003e'table'-\u003e'mutations'\\n       )\\n       \u003e 0\\n\\t\\tFROM system.descriptor\\n\\t  WHERE id = $1::REGCLASS\\n\\t\\t\" [\"public.table180\"]: can't scan into dest[0]: cannot assign NULL to *bool"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 3cf3b0ea3b08cb24e4d6b84c6c237b856ce6b411:

          |  "expectedExecErrors": "3F000,42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema60.seq61\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:52:14.580527",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER INDEX public.table44@\"index44_63\" RENAME TO \"index44_64\""
          |  ],
          |  "expectedExecErrors": "42704",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: index \"index44_63\" does not exist (SQLSTATE 42704)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:52:14.600418",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table44 SET LOCALITY REGIONAL BY ROW"
          |  ],
          |  "expectedExecErrors": "42P16",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot alter a table's LOCALITY if its database is not multi-region enabled (SQLSTATE 42P16)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:52:14.460483",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table44 DROP CONSTRAINT \"table44_col44_57_col44_58_col44_54_col44_47_col44_48_key\""
          |  ],
          |  "expectedExecErrors": "0A000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: unimplemented: cannot drop UNIQUE constraint \"table44_col44_57_col44_58_col44_54_col44_47_col44_48_key\" using ALTER TABLE DROP CONSTRAINT, use DROP INDEX CASCADE instead (SQLSTATE 0A000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:52:14.411899",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange ADD REGION \"us-east1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot add region \"us-east1\" to database schemachange (SQLSTATE 42P12)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 2ef24098cc88bab3958ad905f46b0ed7e74bd7db:

          |  "workerId": 0,
          |  "clientTimestamp": "12:29:51.433756",
          |  "ops": [
          |   "BEGIN",
          |   "DROP SCHEMA \"public\" CASCADE"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot drop schema \"public\" (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:29:51.108787",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"europe-west2\"",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=1f64a6b3 key=/Table/SystemConfigSpan/Start pri=0.07927013 epo=0 ts=1634041791.897166886,1 min=1634041791.108659447,0 seq=24} lock=true stat=PENDING rts=1634041791.108659447,0 wto=false gul=1634041791.608659447,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:29:51.897288",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TYPE schema1.enum106 AS ENUM ('asaad', 'afdfa', 'fdf', 'aasfsa', 'faf')",
          |   "ALTER DATABASE schemachange ADD REGION \"us-east1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot add region \"us-east1\" to database schemachange (SQLSTATE 42P12)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:29:51.435536",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE IF NOT EXISTS public.table80 (col80_81 STRING, col80_82 STRING NOT NULL, col80_83 TIMETZ NOT NULL, col80_84 CHAR, col80_85 GEOGRAPHY NULL, col80_86 BIT(9) NOT NULL, col80_87 INT4 NULL, col80_88 BYTES, col80_89 INT4 NULL, col80_90 BIT(48) NULL, col80_91 TIME NULL, col80_92 STRING NULL AS (lower(CAST(col80_85 AS STRING))) VIRTUAL, col80_93 STRING NOT NULL AS (lower(CAST(col80_86 AS STRING))) STORED, col80_94 INT4 NULL AS (col80_89 + col80_87) STORED, col80_95 STRING NOT NULL AS (lower(col80_82)) VIRTUAL, col80_96 STRING NOT NULL AS (lower(CAST(col80_83 AS STRING))) VIRTUAL, INDEX (col80_81, (col80_87 + col80_89) DESC), INDEX (col80_87 ASC, col80_95 DESC, col80_93, col80_88 ASC, col80_86 DESC) STORING (col80_81, col80_83), INVERTED INDEX (col80_81, col80_83 ASC, col80_85 DESC), UNIQUE (col80_91 ASC, col80_88 ASC, col80_95), INDEX (col80_82 DESC, col80_96), UNIQUE (col80_83 ASC, col80_90 DESC, col80_95 DESC) STORING (col80_82, col80_84, col80_86, col80_88, col80_91), UNIQUE (col80_94 ASC, (CASE WHEN col80_88 IS NULL THEN ' C':::STRING ELSE '$':::STRING END), col80_95, (col80_89 + col80_87), col80_82 ASC, col80_92 ASC, col80_81 ASC, (col80_89 + col80_87)))",
          |   "CREATE TYPE public.enum102 AS ENUM ('sdssfs', 'afsdda', 'aaaf', 'ffaa')",
          |   "ALTER TABLE public.table80 RENAME COLUMN \"crdb_internal_idx_expr_1\" TO \"col80_96\""
          |  ],
          |  "expectedExecErrors": "42701",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: column \"crdb_internal_idx_expr_1\" is inaccessible and cannot be renamed (SQLSTATE 42703)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ e67cf963427faf93670a4c87e0d02bc0ef213b1d:

          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:03:16.046695",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER INDEX schema25.table26@\"IrrelevantConstraintName\" RENAME TO \"OtherConstraintName\""
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema25.table26\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:03:16.107562",
          |  "ops": [
          |   "BEGIN",
          |   "DROP SCHEMA \"public\" CASCADE"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot drop schema \"public\" (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:03:16.139938",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": ""
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:03:15.903899",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-west1\"",
          |   "ALTER DATABASE schemachange ADD REGION \"us-east1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "***FAIL; Failed to receive an execution error when errors were expected"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 3 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 2712f35c0c22919ed5547f9f64c03e907373e74f:

          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=6f16e5a5 key=/Table/SystemConfigSpan/Start pri=0.00114371 epo=0 ts=1634212375.143186533,1 min=1634212374.965030540,0 seq=7} lock=true stat=PENDING rts=1634212374.965030540,0 wto=false gul=1634212375.465030540,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:52:55.08632",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table76 ALTER COLUMN \"col76_79\" DROP STORED"
          |  ],
          |  "expectedExecErrors": "42611",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: column \"col76_79\" is not a computed column (SQLSTATE 42611)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:52:54.946013",
          |  "ops": [
          |   "BEGIN",
          |   "DROP TABLE IF EXISTS public.table76 CASCADE",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=5176c4fb key=/Table/SystemConfigSpan/Start pri=0.00945530 epo=0 ts=1634212375.260893667,1 min=1634212374.945892901,0 seq=13} lock=true stat=PENDING rts=1634212374.945892901,0 wto=false gul=1634212375.445892901,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:52:54.874034",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE VIEW public.view107 AS SELECT public.table76.col76_87, public.table76.col76_88, public.table76.col76_92, public.table76.col76_79, public.table76.col76_85, public.table76.col76_82, public.table76.col76_89, public.table76.col76_91, public.table76.col76_77, public.table76.col76_82, public.table76.col76_92, public.table76.col76_79 FROM public.table76, public.table76"
          |  ],
          |  "expectedExecErrors": "42701,42712",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: source name \"table76\" specified more than once (missing AS clause) (SQLSTATE 42712)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:52:54.874141",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table76 ADD COLUMN col76_97 NAME"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: Validation FAIL:\nid 68, db schemachange, schema public, name table76: mutation job 701700424787197953: job not found"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ b3c1760a030791143c57e89c73d1f26ae8babaf5:

          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:777
          | runtime.goexit
          |     /usr/local/go/src/runtime/asm_amd64.s:1371
        Wraps: (2) output in run_120826.018452258_n1_workload_run_schemachange
        Wraps: (3) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-3593071-1634365596-74-n3cpu4-geo:1 -- ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 --txn-log /mnt/data1/cockroach/transactions.json returned
          | stderr:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          |   pgx logger [error]: Exec logParams=map[args:[] err:ERROR: cannot add region "us-west1" to database schemachange (SQLSTATE 42P12) sql:ALTER DATABASE schemachange ADD REGION "us-west1"]
          | I211016 12:08:28.073199 237 workload/pgx_helpers.go:61  [-] 47  pgx logger [error]: Exec logParams=map[args:[] err:ERROR: cannot drop schema "public" (SQLSTATE 3F000) sql:DROP SCHEMA "public" CASCADE]
          | I211016 12:08:28.088945 43 workload/pgx_helpers.go:61  [-] 48  pgx logger [error]: Exec logParams=map[args:[] err:ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=28801e47 key=/Table/SystemConfigSpan/Start pri=0.00848961 epo=0 ts=1634386107.757724833,1 min=1634386107.740213436,0 seq=7} lock=true stat=PENDING rts=1634386107.740213436,0 wto=false gul=1634386108.240213436,0 (SQLSTATE 40001) sql:commit]
          | I211016 12:08:28.136969 44 workload/pgx_helpers.go:61  [-] 49  pgx logger [error]: Query logParams=map[args:[] err:ERROR: column "crdb_internal_idx_expr_2" does not exist (SQLSTATE 42703) sql:
          | I211016 12:08:28.136969 44 workload/pgx_helpers.go:61  [-] 49 +     SELECT (
          | I211016 12:08:28.136969 44 workload/pgx_helpers.go:61  [-] 49 +        SELECT count(*)
          | I211016 12:08:28.136969 44 workload/pgx_helpers.go:61  [-] 49 +          FROM (
          | I211016 12:08:28.136969 44 workload/pgx_helpers.go:61  [-] 49 +                SELECT DISTINCT col89_103, col89_90, col89_101, crdb_internal_idx_expr_2, col89_94, col89_92, crdb_internal_idx_expr
          | I211016 12:08:28.136969 44 workload/pgx_helpers.go:61  [-] 49 +                  FROM schema2.table89
          | I211016 12:08:28.136969 44 workload/pgx_helpers.go:61  [-] 49 +                 WHERE col89_103 IS NOT NULL OR col89_90 IS NOT NULL OR col89_101 IS NOT NULL OR crdb_internal_idx_expr_2 IS NOT NULL OR col89_94 IS NOT NULL OR col89_92 IS NOT NULL OR crdb_internal_idx_expr IS NOT NULL 
          | I211016 12:08:28.136969 44 workload/pgx_helpers.go:61  [-] 49 +               )
          | I211016 12:08:28.136969 44 workload/pgx_helpers.go:61  [-] 49 +       )
          | I211016 12:08:28.136969 44 workload/pgx_helpers.go:61  [-] 49 +       = (
          | I211016 12:08:28.136969 44 workload/pgx_helpers.go:61  [-] 49 +         SELECT count(*)
          | I211016 12:08:28.136969 44 workload/pgx_helpers.go:61  [-] 49 +           FROM schema2.table89
          | I211016 12:08:28.136969 44 workload/pgx_helpers.go:61  [-] 49 +          WHERE col89_103 IS NOT NULL OR col89_90 IS NOT NULL OR col89_101 IS NOT NULL OR crdb_internal_idx_expr_2 IS NOT NULL OR col89_94 IS NOT NULL OR col89_92 IS NOT NULL OR crdb_internal_idx_expr IS NOT NULL 
          | I211016 12:08:28.136969 44 workload/pgx_helpers.go:61  [-] 49 +        );
          | I211016 12:08:28.136969 44 workload/pgx_helpers.go:61  [-] 49 + ]
          | Error: ***UNEXPECTED ERROR; Failed to generate a random operation: scanBool: "\n\t\tSELECT (\n\t       SELECT count(*)\n\t         FROM (\n\t               SELECT DISTINCT col89_103, col89_90, col89_101, crdb_internal_idx_expr_2, col89_94, col89_92, crdb_internal_idx_expr\n\t                 FROM schema2.table89\n\t                WHERE col89_103 IS NOT NULL OR col89_90 IS NOT NULL OR col89_101 IS NOT NULL OR crdb_internal_idx_expr_2 IS NOT NULL OR col89_94 IS NOT NULL OR col89_92 IS NOT NULL OR crdb_internal_idx_expr IS NOT NULL \n\t              )\n\t      )\n\t      = (\n\t        SELECT count(*)\n\t          FROM schema2.table89\n\t         WHERE col89_103 IS NOT NULL OR col89_90 IS NOT NULL OR col89_101 IS NOT NULL OR crdb_internal_idx_expr_2 IS NOT NULL OR col89_94 IS NOT NULL OR col89_92 IS NOT NULL OR crdb_internal_idx_expr IS NOT NULL \n\t       );\n\t" []: ERROR: column "crdb_internal_idx_expr_2" does not exist (SQLSTATE 42703)
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ``````
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 --txn-log /mnt/data1/cockroach/transactions.json
          |   | ``````
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | ,
          |  "clientTimestamp": "12:08:27.747557",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE public.table80 (col80_81 TIMETZ NOT NULL, col80_82 BIT(2) NOT NULL, col80_83 REGCLASS NOT NULL, col80_84 REGROLE NOT NULL, col80_85 REGPROC NOT NULL, PRIMARY KEY (col80_82 DESC, col80_85, col80_81, col80_84 DESC), col80_86 STRING NOT NULL AS (lower(CAST(col80_81 AS STRING))) STORED, col80_87 STRING NOT NULL AS (lower(CAST(col80_84 AS STRING))) STORED, col80_88 STRING NOT NULL AS (lower(CAST(col80_84 AS STRING))) STORED, INDEX (col80_83 DESC), UNIQUE (col80_82 DESC, col80_86, col80_87 ASC, col80_83 DESC, col80_88, col80_85, col80_81 ASC), UNIQUE (col80_88 DESC, col80_83 DESC, col80_87 DESC, col80_84 ASC, col80_85), UNIQUE (col80_88 DESC, col80_84 DESC, col80_83, col80_87, col80_81, col80_85 DESC, col80_82 ASC), UNIQUE (col80_82, col80_85, col80_83 DESC) STORING (col80_87), UNIQUE (col80_86), INDEX (col80_81 DESC, col80_83 DESC, col80_84, col80_87 DESC), UNIQUE (col80_88 ASC, col80_81, col80_85 DESC), FAMILY (col80_81), FAMILY (col80_82), FAMILY (col80_85, col80_86, col80_87), FAMILY (col80_83, col80_88), FAMILY (col80_84))",
          |   "CREATE TABLE schema2.table89 (col89_90 NAME NOT NULL, col89_91 TIMESTAMP, col89_92 INTERVAL NULL, col89_93 FLOAT8, col89_94 VARCHAR NULL, col89_95 INT2, col89_96 TIME, col89_97 CHAR NULL, col89_98 \"char\" NOT NULL, col89_99 REGTYPE, col89_100 STRING NULL AS (lower(col89_97)) VIRTUAL, col89_101 FLOAT8 AS (col89_93 + '-Inf':::FLOAT8) VIRTUAL, col89_102 FLOAT8 AS (col89_93 + (-2.772854708133192):::FLOAT8) VIRTUAL, col89_103 STRING NULL AS (lower(col89_97)) STORED, col89_104 STRING NOT NULL AS (lower(col89_98)) VIRTUAL, col89_105 STRING NULL AS (lower(CAST(col89_92 AS STRING))) STORED, col89_106 INT2 AS (col89_95 + (-27514):::INT8) VIRTUAL, col89_107 STRING NOT NULL AS (lower(col89_90)) VIRTUAL, UNIQUE (col89_97, col89_99 DESC, col89_104, col89_106 DESC) STORING (col89_91, col89_95, col89_98, col89_105), UNIQUE (col89_107, col89_102, lower(col89_90) DESC, col89_94 DESC, col89_95) STORING (col89_91, col89_93, col89_103, col89_105), UNIQUE (col89_106 DESC, col89_93 ASC, col89_99 ASC, col89_94 ASC, col89_96, col89_91, col89_92, col89_101 DESC, col89_90 ASC, col89_104 ASC, col89_107), INDEX (col89_93 ASC, col89_92, col89_94 ASC, col89_107 ASC, col89_91 ASC, col89_103 DESC, lower(CAST(col89_96 AS STRING)) DESC, col89_96 ASC, col89_101, col89_100 ASC, col89_104, col89_105 ASC, lower(col89_97) ASC) STORING (col89_90, col89_98, col89_99), INDEX (col89_101, col89_104 ASC, col89_96 ASC, col89_102 DESC, col89_103, col89_98, col89_91, col89_107, col89_100, col89_97 ASC, col89_94 ASC, col89_105 ASC, col89_106) STORING (col89_93, col89_95), INDEX (col89_97, col89_90, col89_102 ASC, col89_107, col89_103 DESC, col89_91, col89_92 ASC, col89_104, col89_98) STORING (col89_94, col89_95, col89_96, col89_99), INDEX (col89_91 DESC, col89_90, col89_97, col89_104 DESC, col89_105, col89_102, col89_101, col89_103 ASC, col89_98 DESC, col89_94 ASC, lower(CAST(col89_99 AS STRING)) DESC, col89_107 ASC, lower(col89_98)))"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: scanBool: \"\\n\\t\\tSELECT (\\n\\t       SELECT count(*)\\n\\t         FROM (\\n\\t               SELECT DISTINCT col89_103, col89_90, col89_101, crdb_internal_idx_expr_2, col89_94, col89_92, crdb_internal_idx_expr\\n\\t                 FROM schema2.table89\\n\\t                WHERE col89_103 IS NOT NULL OR col89_90 IS NOT NULL OR col89_101 IS NOT NULL OR crdb_internal_idx_expr_2 IS NOT NULL OR col89_94 IS NOT NULL OR col89_92 IS NOT NULL OR crdb_internal_idx_expr IS NOT NULL \\n\\t              )\\n\\t      )\\n\\t      = (\\n\\t        SELECT count(*)\\n\\t          FROM schema2.table89\\n\\t         WHERE col89_103 IS NOT NULL OR col89_90 IS NOT NULL OR col89_101 IS NOT NULL OR crdb_internal_idx_expr_2 IS NOT NULL OR col89_94 IS NOT NULL OR col89_92 IS NOT NULL OR crdb_internal_idx_expr IS NOT NULL \\n\\t       );\\n\\t\" []: ERROR: column \"crdb_internal_idx_expr_2\" does not exist (SQLSTATE 42703)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ cb725a6acdc08257ad81273535dadad5d0a727dc:

          |  "clientTimestamp": "11:52:11.352624",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq69",
          |   "CREATE INDEX \"IrrelevantName\" ON schema1.table88 (\"IrrelevantColumn\" ASC)"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema1.table88\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:52:11.103093",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TYPE public.enum56 AS ENUM ('sda', 'dfddas', 'sdss')",
          |   "ALTER INDEX schema1.table87@\"IrrelevantConstraintName\" RENAME TO \"OtherConstraintName\""
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema1.table87\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:52:10.942907",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "CREATE TYPE public.enum86 AS ENUM ('faa', 'afasda', 'f', 'ff', 'aaasf')",
          |   "ALTER DATABASE schemachange ADD REGION \"us-east1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot add region \"us-east1\" to database schemachange (SQLSTATE 42P12)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:52:11.317184",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE"
          |  ],
          |  "expectedExecErrors": "42602",
          |  "expectedCommitErrors": "",
          |  "message": "***FAIL; Failed to receive an execution error when errors were expected"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ ff34ea874d5c7efc47922f73b7a1d5ee1ef614fc:

          |  "workerId": 0,
          |  "clientTimestamp": "11:45:01.725044",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE"
          |  ],
          |  "expectedExecErrors": "42602",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: database must have associated regions before a survival goal can be set (SQLSTATE 42602)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:45:01.725142",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "DROP SCHEMA \"public\" CASCADE"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot drop schema \"public\" (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:45:01.724967",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "CREATE SCHEMA IF NOT EXISTS schema23 AUTHORIZATION root",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=d66da74a key=/Table/SystemConfigSpan/Start pri=0.00281734 epo=0 ts=1634730302.018326489,1 min=1634730301.722878203,0 seq=15} lock=true stat=PENDING rts=1634730301.722878203,0 wto=false gul=1634730302.222878203,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:45:01.725178",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"europe-west2\"",
          |   "ALTER DATABASE schemachange ADD REGION \"europe-west2\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: region \"europe-west2\" already added to database (SQLSTATE 42710)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ c5a6b266917ee3846dbd7ae1126c6a5d55cf439b:

          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"europe-west2\"",
          |   "ALTER TABLE public.table48 ALTER COLUMN IrrelevantColumnName SET DEFAULT \"IrrelevantValue\""
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.table48\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:00:23.366056",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA schema13 AUTHORIZATION root",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=91483708 key=/Table/SystemConfigSpan/Start pri=0.00909758 epo=0 ts=1634817623.676862592,1 min=1634817623.365972924,0 seq=15} lock=true stat=PENDING rts=1634817623.365972924,0 wto=false gul=1634817623.865972924,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:00:23.366966",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "CREATE TYPE public.enum49 AS ENUM ('ssffad', 'fsdsf', 'adfsfs', 'sasdf', 'sdf', 'dds')",
          |   "DROP SCHEMA \"public\" CASCADE"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot drop schema \"public\" (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:00:23.368262",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE IF NOT EXISTS public.table4 (col4_5 REGTYPE NULL, col4_6 INT2 NULL, col4_7 INT8 NULL, col4_8 INTERVAL NULL, col4_9 REGNAMESPACE, col4_10 TIMETZ NULL, col4_11 INT2 NULL AS (col4_6 + 6046:::INT8) STORED, INDEX (col4_10 ASC, lower(CAST(col4_8 AS STRING)) DESC, col4_6 ASC, (col4_7 + col4_6), (col4_6 + col4_7) ASC, col4_9 DESC) STORING (col4_7, col4_11))",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-west1\"",
          |   "ALTER DATABASE schemachange ADD REGION \"europe-west2\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "***FAIL; Failed to receive an execution error when errors were expected"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 7d5dea07c21e223fbfd8d5377f1d7132c88baf26:

          |   "CREATE TABLE public.table76 (col76_77 INTERVAL NOT NULL, col76_78 TIMETZ NOT NULL, col76_79 TIME NOT NULL, col76_80 GEOGRAPHY NULL, col76_81 REGCLASS NOT NULL, col76_82 BOX2D NOT NULL, col76_83 REGROLE NULL, col76_84 REGROLE NULL, col76_85 BOOL NOT NULL, col76_86 TIMETZ NOT NULL, col76_87 STRING NOT NULL, col76_88 DATE[] NULL, col76_89 REGROLE NOT NULL, PRIMARY KEY (col76_82 DESC, col76_86 ASC, col76_85, col76_87 DESC))",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=2deb877f key=/Table/SystemConfigSpan/Start pri=0.02847221 epo=0 ts=1634903489.645504374,1 min=1634903489.376683421,0 seq=7} lock=true stat=PENDING rts=1634903489.376683421,0 wto=false gul=1634903489.876683421,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:51:29.377496",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange ADD REGION \"us-east1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot add region \"us-east1\" to database schemachange (SQLSTATE 42P12)"
          | }
          | _elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
          |     1.0s        0           36.6           41.7     22.0    218.1    302.0    302.0 opOk
          |     1.0s        0            5.2            6.0    335.5    520.1    520.1    520.1 txnCmtErr
          |     1.0s        0            4.4            5.0    268.4    704.6    704.6    704.6 txnOk
          |     1.0s        0           34.9           39.7      3.1     54.5    117.4    117.4 txnRbk
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:51:29.629734",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "CREATE TYPE public.enum107 AS ENUM ('daafss', 'f', 'fad')",
          |   "ALTER VIEW schema1.view114 RENAME TO schema1.view115"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema1.view114\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:51:29.58477",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE IF NOT EXISTS schema1.table98 (col98_99 \"char\", col98_100 REGNAMESPACE, col98_101 INT4 NOT NULL, col98_102 VARCHAR NOT NULL, col98_103 INTERVAL[] NULL, col98_104 STRING NOT NULL AS (lower(col98_102)) VIRTUAL, col98_105 STRING AS (lower(col98_99)) VIRTUAL, INDEX (col98_105 DESC, col98_102 ASC, lower(col98_99), (CASE WHEN col98_103 IS NULL THEN '':::STRING ELSE e'G\\b\\x01\\x1c4iC\\x1a':::STRING END) DESC, col98_100) STORING (col98_99, col98_101, col98_103), INDEX (col98_99 ASC, col98_104 ASC), INDEX (lower(col98_99) ASC, col98_101 ASC) STORING (col98_99, col98_100, col98_102, col98_103))",
          |   "CREATE INDEX IF NOT EXISTS index98_117 ON schema1.table98 (col98_102 ASC, crdb_internal_idx_expr DESC, crdb_internal_idx_expr_1 ASC, crdb_internal_idx_expr_2, col98_100, col98_99) STORING (col98_105, col98_104)"
          |  ],
          |  "expectedExecErrors": "XXUUU",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: column \"crdb_internal_idx_expr\" is inaccessible and cannot be referenced (SQLSTATE 42703)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 360956c1f727c600cbec55f457dd013d5fb6850b:

          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          |  NULL, col88_162 NAME NULL, col88_163 DECIMAL NOT NULL, col88_164 TIMESTAMPTZ NOT NULL, col88_165 FLOAT8 NOT NULL, PRIMARY KEY (col88_161 ASC, col88_160, col88_159 ASC, col88_164 ASC), col88_166 STRING NOT NULL AS (lower(col88_161)) STORED, col88_167 STRING NOT NULL AS (lower(col88_160)) STORED, col88_168 FLOAT8 NOT NULL AS (col88_165 + (-0.6697092307748731):::FLOAT8) STORED, col88_169 STRING NOT NULL AS (lower(CAST(col88_159 AS STRING))) VIRTUAL, col88_170 STRING NOT NULL AS (lower(col88_161)) VIRTUAL, UNIQUE (col88_164, col88_166, col88_170 ASC, (col88_165 + 0.006480558111155582:::FLOAT8), col88_160 ASC, col88_167 ASC), UNIQUE (col88_161, col88_168 DESC, col88_165 ASC, col88_164 ASC, col88_159, col88_170 DESC, col88_162, col88_169 ASC, col88_160, col88_167 ASC), INDEX (col88_160 DESC), INDEX (col88_159 ASC, col88_168 ASC, col88_165 ASC, col88_167 DESC, col88_169 ASC), INDEX (col88_159 ASC, col88_170 ASC, col88_162 DESC, col88_164 ASC, col88_163 DESC, lower(CAST(col88_159 AS STRING)) DESC, (col88_165 + 0.758031433762377:::FLOAT8), (col88_163 + 2075279130523.344754:::DECIMAL), lower(col88_160) ASC, col88_160) STORING (col88_165, col88_166), UNIQUE (col88_160, col88_164 DESC, col88_163 DESC), INDEX (col88_162 ASC, col88_163 ASC, col88_160 ASC, col88_159 ASC, col88_169 ASC, col88_170 DESC, col88_164, col88_168 DESC, col88_165 DESC, col88_166 DESC), UNIQUE (col88_162 ASC, col88_163 ASC, lower(CAST(col88_159 AS STRING)), lower(CAST(col88_159 AS STRING)) DESC, col88_160 DESC, col88_168, col88_169, col88_167 ASC, col88_170, col88_166 DESC, col88_164), INDEX (col88_170 ASC, col88_165 ASC, col88_168 DESC, col88_164, col88_163 ASC, col88_159 DESC, col88_169 DESC, col88_162, col88_161 ASC, lower(col88_162) DESC, col88_167 ASC), FAMILY (col88_160), FAMILY (col88_167), FAMILY (col88_163, col88_161, col88_162, col88_166, col88_159), FAMILY (col88_165), FAMILY (col88_164, col88_168))",
          |   "CREATE SEQUENCE IF NOT EXISTS schema151.seq172 OWNED BY public.table88.col88_100",
          |   "ALTER TABLE public.table88 RENAME COLUMN \"col88_101\" TO \"col88_182\"",
          |   "DROP TABLE public.table88 CASCADE",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=965d8499 key=/Table/SystemConfigSpan/Start pri=0.02757818 epo=0 ts=1635249328.218469768,1 min=1635249327.676014083,0 seq=38} lock=true stat=PENDING rts=1635249327.676014083,0 wto=false gul=1635249328.176014083,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:55:28.031345",
          |  "ops": [
          |   "BEGIN",
          |   "DROP TABLE IF EXISTS public.table88 CASCADE",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "DROP SCHEMA \"schema188\" CASCADE"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: unknown schema \"schema188\" (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:55:27.807558",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table88 RENAME TO public.table176",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=54075c39 key=/Table/SystemConfigSpan/Start pri=0.04278952 epo=0 ts=1635249328.218469768,1 min=1635249327.807481408,0 seq=14} lock=true stat=PENDING rts=1635249327.807481408,0 wto=false gul=1635249328.307481408,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:55:27.879041",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table88 ADD COLUMN col88_177 REGROLE NULL UNIQUE"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: Validation FAIL:\nid 89, db schemachange, schema public, name table88: mutation job 705098312733229057: job not found"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 79e5979416cb426092a83beff0be1c20aebf84c6:

          |   "BEGIN",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq203",
          |   "CREATE SCHEMA IF NOT EXISTS schema206 AUTHORIZATION root",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=7cfa7b15 key=/Table/SystemConfigSpan/Start pri=0.02678121 epo=0 ts=1635333618.437080719,1 min=1635333617.945062744,0 seq=23} lock=true stat=PENDING rts=1635333617.945062744,0 wto=false gul=1635333618.445062744,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:20:17.937138",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER DATABASE schemachange SURVIVE REGION FAILURE"
          |  ],
          |  "expectedExecErrors": "22023",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: at least 3 regions are required for surviving a region failure (SQLSTATE 22023)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:20:17.680374",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER SEQUENCE public.seq29 RENAME TO public.seq166",
          |   "DROP SEQUENCE public.seq166",
          |   "ALTER DATABASE schemachange ADD REGION \"europe-west2\"",
          |   "CREATE TYPE public.enum227 AS ENUM ('dfdddf', 'd', 'fa')",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=77e41482 key=/Table/SystemConfigSpan/Start pri=0.02013189 epo=0 ts=1635333618.521411281,1 min=1635333617.680315771,0 seq=42} lock=true stat=PENDING rts=1635333617.680315771,0 wto=false gul=1635333618.180315771,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:20:18.254223",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE IF NOT EXISTS public.table192 (col192_193 TIME NOT NULL, col192_194 REGNAMESPACE NULL, col192_195 REGPROCEDURE, col192_196 INT4 NULL, col192_197 DATE NULL, col192_198 REGPROCEDURE NOT NULL, col192_199 REGPROCEDURE NOT NULL, col192_200 INT2, col192_201 INTERVAL NULL, col192_202 INET, PRIMARY KEY (col192_199), INDEX (col192_197 DESC, col192_202 DESC, col192_200 DESC, col192_198 DESC, col192_195 ASC, col192_193 ASC) STORING (col192_194, col192_196), UNIQUE (lower(CAST(col192_193 AS STRING)), col192_193 ASC) STORING (col192_194, col192_196, col192_197, col192_198, col192_200, col192_201, col192_202), UNIQUE (col192_195 ASC, col192_198 DESC, col192_197) STORING (col192_193, col192_194, col192_196, col192_202), UNIQUE (col192_196, col192_199 DESC, col192_202 ASC, col192_200 DESC, col192_195, col192_201) STORING (col192_193, col192_194, col192_197, col192_198), FAMILY (col192_200, col192_197), FAMILY (col192_199), FAMILY (col192_198), FAMILY (col192_194, col192_195), FAMILY (col192_202, col192_201, col192_193), FAMILY (col192_196))",
          |   "ALTER TABLE public.table192 ALTER COLUMN \"col192_195\" DROP NOT NULL"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: scanBool: \"\\n\\t\\tSELECT json_array_length(\\n        crdb_internal.pb_to_json(\\n            'cockroach.sql.sqlbase.Descriptor',\\n            descriptor\\n        )-\u003e'table'-\u003e'mutations'\\n       )\\n       \u003e 0\\n\\t\\tFROM system.descriptor\\n\\t  WHERE id = $1::REGCLASS\\n\\t\\t\" [\"public.table192\"]: can't scan into dest[0]: cannot assign NULL to *bool"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ f279cc9cf1b28ef4418ece4452714e2b2a4c4445:

          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:43:58.275405",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange ADD REGION \"us-west1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot add region \"us-west1\" to database schemachange (SQLSTATE 42P12)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:43:58.130398",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE IF NOT EXISTS public.table218 (col218_219 DECIMAL, col218_220 JSONB NOT NULL, col218_221 VARCHAR NOT NULL, col218_222 REGTYPE NULL, col218_223 JSONB NULL, col218_224 \"char\" NOT NULL, col218_225 TIMESTAMP, col218_226 TIMESTAMPTZ, col218_227 UUID NOT NULL, col218_228 JSONB NOT NULL, col218_229 TIME NULL, col218_230 REGTYPE, col218_231 GEOGRAPHY, col218_232 STRING NULL, col218_233 STRING NOT NULL AS (lower(CAST(col218_227 AS STRING))) VIRTUAL, col218_234 STRING NULL AS (lower(CAST(col218_222 AS STRING))) STORED, col218_235 DECIMAL AS (col218_219 + (-4.866736150530184254E+29):::DECIMAL) STORED, UNIQUE (col218_221 ASC) STORING (col218_219, col218_220, col218_223, col218_227, col218_228, col218_230, col218_231, col218_235))",
          |   "CREATE TABLE IF NOT EXISTS schema244.table245 (col245_246 VARCHAR, col245_247 \"char\" NULL, col245_248 REGNAMESPACE NOT NULL, col245_249 CHAR NULL, col245_250 TIMESTAMPTZ NOT NULL, col245_251 REGPROCEDURE NULL, col245_252 BOX2D, col245_253 STRING NOT NULL, col245_254 BOX2D NULL, col245_255 REGTYPE NOT NULL, col245_256 INT8[] NULL, col245_257 VARCHAR NOT NULL, col245_258 GEOGRAPHY NOT NULL, col245_259 GEOMETRY NOT NULL, col245_260 FLOAT4 NOT NULL, col245_261 REGPROC NULL, UNIQUE (col245_253 DESC, col245_249 ASC, lower(CAST(col245_248 AS STRING)) ASC, col245_247 DESC), INDEX (lower(col245_249) DESC) STORING (col245_246, col245_248, col245_249, col245_250, col245_252, col245_253, col245_254, col245_255, col245_260, col245_261), UNIQUE (col245_248 DESC, col245_254 DESC) STORING (col245_246, col245_251, col245_256, col245_257, col245_261), UNIQUE (lower(CAST(col245_261 AS STRING)) ASC, col245_252 DESC, col245_251 DESC, col245_260 ASC), FAMILY (col245_258), FAMILY (col245_250, col245_251, col245_256), FAMILY (col245_253, col245_261), FAMILY (col245_260), FAMILY (col245_257), FAMILY (col245_259), FAMILY (col245_247, col245_255, col245_252, col245_254, col245_246, col245_248), FAMILY (col245_249))"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot create \"schema244.table245\" because the target database or schema does not exist (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:43:58.331443",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq212",
          |   "CREATE SCHEMA schema237 AUTHORIZATION root"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_PUSHER_ABORTED): \"sql txn\" meta={id=11cc1e2d key=/Table/SystemConfigSpan/Start pri=0.02280334 epo=0 ts=1635421438.550024416,1 min=1635421438.331385318,0 seq=15} lock=true stat=ABORTED rts=1635421438.331385318,0 wto=false gul=1635421438.831385318,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:43:58.516657",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table169 ALTER COLUMN col169_173 SET DEFAULT 0.2911220602229456:::FLOAT8",
          |   "DROP TABLE public.table169 RESTRICT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: ERROR: relation \"public.table169\" does not exist (SQLSTATE 42P01)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ ab508bd125bcf2fdb791d887e0aa26ebd64a8bcc:

          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "CREATE SCHEMA schema90 AUTHORIZATION root",
          |   "ALTER TABLE schema7.table145 ALTER COLUMN \"IrrelevantColumnName\" DROP NOT NULL"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema7.table145\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:57:01.150544",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"europe-west2\"",
          |   "DROP SCHEMA \"public\" CASCADE"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot drop schema \"public\" (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:57:02.042346",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA IF NOT EXISTS schema131 AUTHORIZATION root",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=81612ea7 key=/Table/SystemConfigSpan/Start pri=0.00522509 epo=0 ts=1635854225.032712907,1 min=1635854222.042286913,0 seq=15} lock=true stat=PENDING rts=1635854222.042286913,0 wto=false gul=1635854222.542286913,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:57:02.130056",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE IF NOT EXISTS public.table113 (col113_114 DECIMAL, col113_115 JSONB NOT NULL, col113_116 OID NOT NULL, col113_117 DECIMAL, col113_118 \"char\", col113_119 TIMESTAMP NULL, col113_120 NAME NOT NULL, col113_121 INTERVAL, col113_122 DECIMAL AS (col113_114 + col113_117) STORED, col113_123 STRING AS (lower(CAST(col113_121 AS STRING))) VIRTUAL, col113_124 STRING AS (lower(CAST(col113_121 AS STRING))) STORED, col113_125 DECIMAL AS (col113_117 + col113_114) STORED, col113_126 DECIMAL AS (col113_114 + col113_117) VIRTUAL, col113_127 DECIMAL AS (col113_114 + col113_117) VIRTUAL, col113_128 STRING NOT NULL AS (lower(CAST(col113_115 AS STRING))) STORED, UNIQUE (col113_116 ASC, lower(col113_120) ASC, col113_125 ASC) STORING (col113_114, col113_115, col113_119, col113_121), INDEX (col113_123 ASC, col113_114 ASC) STORING (col113_116, col113_119, col113_121, col113_124), FAMILY (col113_128), FAMILY (col113_120, col113_122), FAMILY (col113_124, col113_116, col113_119, col113_117, col113_121), FAMILY (col113_118, col113_125), FAMILY (col113_115, col113_114))",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE"
          |  ],
          |  "expectedExecErrors": "42602",
          |  "expectedCommitErrors": "",
          |  "message": "***FAIL; Failed to receive an execution error when errors were expected"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ a35bb95e35ebefb6019382991c7fa534d1d50e44:

          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:07:17.139291",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA IF NOT EXISTS schema24 AUTHORIZATION root"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): \"sql txn\" meta={id=19051852 key=/Table/SystemConfigSpan/Start pri=0.01278650 epo=0 ts=1635941237.382797902,1 min=1635941237.139211406,0 seq=8} lock=true stat=PENDING rts=1635941237.139211406,0 wto=false gul=1635941237.639211406,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:07:17.136506",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE public.table33 (col33_34 INTERVAL NOT NULL, col33_35 JSONB NOT NULL, INDEX (col33_34 ASC) STORING (col33_35), INDEX (col33_34 DESC))",
          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE"
          |  ],
          |  "expectedExecErrors": "42602",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: database must have associated regions before a survival goal can be set (SQLSTATE 42602)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:07:17.36889",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TYPE schema1.enum39 AS ENUM ('aa', 's', 'aaf')",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": ""
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:07:17.137369",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE public.table2 (col2_3 OID NOT NULL, col2_4 REGTYPE NOT NULL, col2_5 DECIMAL NOT NULL, col2_6 INT8 NOT NULL, col2_7 OID, col2_8 REGROLE NOT NULL, col2_9 REGPROCEDURE NULL, col2_10 BIT(14) NOT NULL, col2_11 \"char\" NULL, col2_12 INT8 NOT NULL, PRIMARY KEY (col2_12 ASC, col2_10 DESC, col2_4 DESC, col2_5), col2_13 STRING NOT NULL AS (lower(CAST(col2_10 AS STRING))) VIRTUAL, col2_14 DECIMAL NOT NULL AS (col2_5 + (-5.952699390842781126E+31):::DECIMAL) STORED, col2_15 STRING NULL AS (lower(col2_11)) STORED, col2_16 INT8 NOT NULL AS (col2_12 + (-4100090881495882795):::INT8) VIRTUAL, col2_17 STRING NOT NULL AS (lower(CAST(col2_10 AS STRING))) VIRTUAL, col2_18 STRING NOT NULL AS (lower(CAST(col2_10 AS STRING))) VIRTUAL, INDEX (col2_9, col2_3 DESC, col2_6, col2_18 DESC, col2_11 ASC, col2_14, col2_13 DESC, col2_12 ASC, col2_7, col2_10 DESC, (col2_12 + col2_6) ASC, col2_15 DESC, col2_8, col2_16 ASC), INDEX (col2_13 ASC, col2_9 DESC, lower(CAST(col2_7 AS STRING)) ASC, col2_16), INDEX (col2_16 DESC, col2_8 ASC, col2_9 DESC, col2_18 ASC, col2_13 DESC, col2_10, col2_14 ASC, col2_11 ASC, col2_4 DESC, (col2_6 + col2_12) ASC, col2_12 DESC, col2_15 DESC, col2_6, col2_5 DESC, col2_3))",
          |   "CREATE INDEX index2_41 ON public.table2 (col2_5 DESC, col2_6 DESC, crdb_internal_idx_expr_1 DESC, col2_8 DESC, col2_18 DESC, col2_17 DESC, col2_14 ASC, col2_13, col2_3 ASC, crdb_internal_idx_expr ASC) STORING (col2_4, col2_11, col2_12, col2_16, col2_9, crdb_internal_idx_expr_2, col2_10, col2_7)"
          |  ],
          |  "expectedExecErrors": "42701,XXUUU",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: column \"crdb_internal_idx_expr_1\" is inaccessible and cannot be referenced (SQLSTATE 42703)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 80d2fb8e621d25c2ceb89562e45b63205a1c36e9:

          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot drop schema \"public\" (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "13:26:23.264882",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": ""
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "13:26:23.382474",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange ADD REGION \"us-west1\""
          |  ],
          |  "expectedExecErrors": "42710",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: region \"us-west1\" already added to database (SQLSTATE 42710)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "13:26:23.402136",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq87",
          |   "ALTER TABLE public.table81 RENAME TO schema115.table116"
          |  ],
          |  "expectedExecErrors": "3F000,42602",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot create \"schema115.table116\" because the target database or schema does not exist (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "13:26:23.630087",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table81 ALTER COLUMN \"crdb_internal_idx_expr_1\" DROP STORED"
          |  ],
          |  "expectedExecErrors": "42611",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: column \"crdb_internal_idx_expr_1\" is not a stored computed column (SQLSTATE 42611)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 97f2e7dbe93a3873916b99307f1372d7c095347c:

          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE"
          |  ],
          |  "expectedExecErrors": "42602",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: database must have associated regions before a survival goal can be set (SQLSTATE 42602)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "13:25:33.415609",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA schema57 AUTHORIZATION root",
          |   "CREATE SEQUENCE IF NOT EXISTS schema57.seq121",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=3503f50a key=/Table/SystemConfigSpan/Start pri=0.03626766 epo=0 ts=1637069133.867852750,1 min=1637069133.415543598,0 seq=23} lock=true stat=PENDING rts=1637069133.415543598,0 wto=false gul=1637069133.915543598,0 (SQLSTATE 40001)"
          | }
          | _elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
          |     1.0s        0           38.1           41.0      7.9    402.7    486.5    486.5 opOk
          |     1.0s        0            5.6            6.0    352.3    570.4    570.4    570.4 txnCmtErr
          |     1.0s        0            4.6            5.0    260.0    536.9    536.9    536.9 txnOk
          |     1.0s        0           34.4           37.0      2.0     13.6     18.9     18.9 txnRbk
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "13:25:33.445033",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-west1\"",
          |   "DROP SCHEMA \"public\" CASCADE"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot drop schema \"public\" (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "13:25:33.405058",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"europe-west2\"",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\""
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: region \"us-east1\" has not been added to the database (SQLSTATE 42602)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 774ea54918869b6ba51674942fe5e9f81327d2a7:

          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=b9f4f288 key=/Table/SystemConfigSpan/Start pri=0.00744853 epo=0 ts=1637151149.042207851,1 min=1637151148.541006651,0 seq=37} lock=true stat=PENDING rts=1637151148.541006651,0 wto=false gul=1637151149.041006651,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:12:28.794023",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA schema62 AUTHORIZATION root",
          |   "ALTER TABLE public.table50 RENAME COLUMN \"col50_56\" TO \"col50_69\""
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_PUSHER_ABORTED): \"sql txn\" meta={id=c5ec0ae2 key=/Table/SystemConfigSpan/Start pri=0.00740862 epo=0 ts=1637151149.043453934,1 min=1637151148.793977964,0 seq=15} lock=true stat=ABORTED rts=1637151148.793977964,0 wto=false gul=1637151149.293977964,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:12:29.521033",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange ADD REGION \"europe-west2\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot add region \"europe-west2\" to database schemachange (SQLSTATE 42P12)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:12:29.264898",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE"
          |  ],
          |  "expectedExecErrors": "42602",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: database must have associated regions before a survival goal can be set (SQLSTATE 42602)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:12:28.802243",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table50 ADD COLUMN col50_66 BOOL UNIQUE",
          |   "CREATE TYPE public.enum76 AS ENUM ('dfdfsf', 'aafafs', 'aass', 'fafasa')"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: Validation FAIL:\nid 75, db schemachange, schema public, name table50: mutation job 711330199116283905: job not found"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ bd65dbb707ef3a852d89a4914610cb3e72c6b657:

          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | S ENUM ('fa', 'dsasa', 'aadfad', 'dds')",
          |   "CREATE TABLE IF NOT EXISTS public.table245 (col245_246 FLOAT4 NOT NULL, col245_247 DATE NOT NULL, col245_248 TIME NOT NULL, col245_249 DECIMAL NOT NULL, col245_250 REGCLASS NOT NULL, col245_251 REGPROC NOT NULL, col245_252 REGCLASS NOT NULL, col245_253 INT2 NOT NULL, PRIMARY KEY (col245_248 ASC, col245_250 ASC, col245_253 DESC, col245_252 ASC, col245_251 ASC, col245_246, col245_247 DESC, col245_249 DESC), col245_254 FLOAT4 NOT NULL AS (col245_246 + 0.0:::FLOAT8) VIRTUAL, col245_255 STRING NOT NULL AS (lower(CAST(col245_250 AS STRING))) VIRTUAL, col245_256 STRING NOT NULL AS (lower(CAST(col245_248 AS STRING))) STORED, col245_257 INT2 NOT NULL AS (col245_253 + 17193:::INT8) VIRTUAL, col245_258 STRING NOT NULL AS (lower(CAST(col245_247 AS STRING))) VIRTUAL, UNIQUE (col245_247 ASC, col245_254, col245_255 DESC, lower(CAST(col245_252 AS STRING)), col245_246, col245_256), UNIQUE (col245_250 DESC, col245_249 DESC, col245_256, col245_254 ASC, col245_257 DESC, col245_252 ASC, col245_258 DESC), INDEX (lower(CAST(col245_252 AS STRING)) ASC, col245_253 ASC, col245_249 ASC, (col245_249 + (-4.656955735877391038E+24):::DECIMAL) DESC, col245_258, col245_254, col245_247 DESC, col245_257), UNIQUE (col245_248 ASC, col245_258 ASC, col245_252 ASC, col245_253, col245_250 ASC, col245_254 DESC, col245_247 ASC, col245_257, col245_249, col245_256 DESC), UNIQUE (col245_253 ASC, col245_252 DESC, col245_247, col245_257 ASC, col245_258, col245_251 DESC, lower(CAST(col245_247 AS STRING)) ASC, col245_246 DESC, col245_250, col245_254 DESC, (col245_246 + 0.8701131939888:::FLOAT8) ASC, col245_248 ASC) STORING (col245_256), UNIQUE (col245_251 DESC, col245_258 DESC, col245_257 DESC, col245_254 ASC, col245_250 ASC, col245_252 ASC, col245_246, col245_249 ASC, col245_248, col245_256, col245_253 ASC, col245_247, col245_255 DESC), INDEX (col245_257))",
          |   "CREATE SCHEMA IF NOT EXISTS schema263 AUTHORIZATION root",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=a566525a key=/Table/SystemConfigSpan/Start pri=0.02961098 epo=0 ts=1637239818.094398762,1 min=1637239817.779155975,0 seq=47} lock=true stat=PENDING rts=1637239817.779155975,0 wto=false gul=1637239818.279155975,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:50:17.533153",
          |  "ops": [
          |   "BEGIN",
          |   "DROP SEQUENCE IF EXISTS public.seq234",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER TABLE public.table265 ALTER COLUMN IrrelevantColumnName SET NOT NULL"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.table265\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:50:17.749974",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange SURVIVE REGION FAILURE"
          |  ],
          |  "expectedExecErrors": "22023",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: at least 3 regions are required for surviving a region failure (SQLSTATE 22023)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:50:17.442618",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE IF NOT EXISTS public.table237 (col237_238 TIMESTAMP[] NOT NULL, INDEX ((CASE WHEN col237_238 IS NULL THEN e'_zzHi\u003cP\\n':::STRING ELSE e'\\x19uh\u003eB 8{':::STRING END) DESC), INVERTED INDEX (col237_238 DESC), INDEX ((CASE WHEN col237_238 IS NULL THEN e'(!\\x01n\u003e{a':::STRING ELSE '2l4kr':::STRING END) DESC) STORING (col237_238), INVERTED INDEX (col237_238 ASC))",
          |   "ALTER TABLE public.table237 RENAME COLUMN \"crdb_internal_idx_expr_1\" TO \"col237_262\""
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: column \"crdb_internal_idx_expr_1\" is inaccessible and cannot be renamed (SQLSTATE 42703)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 6efca5d977ab81c2266407d59b36ad47fef3078a:

          |  "ops": [
          |   "BEGIN",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq130",
          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE"
          |  ],
          |  "expectedExecErrors": "42602",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: database must have associated regions before a survival goal can be set (SQLSTATE 42602)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:53:27.864288",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA IF NOT EXISTS public AUTHORIZATION root",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": ""
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:53:27.888379",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SEQUENCE IF NOT EXISTS schema65.seq113",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER DATABASE schemachange ADD REGION \"us-west1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot add region \"us-west1\" to database schemachange (SQLSTATE 42P12)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:53:27.937257",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "CREATE TABLE IF NOT EXISTS public.table134 (col134_135 INET NOT NULL, col134_136 FLOAT4 NOT NULL, col134_137 JSONB NOT NULL, col134_138 REGCLASS NOT NULL, col134_139 BYTES, col134_140 CHAR NULL, col134_141 REGCLASS NULL, PRIMARY KEY (col134_138 ASC, col134_135), col134_142 STRING NULL AS (lower(col134_140)) VIRTUAL, col134_143 STRING AS (CASE WHEN col134_139 IS NULL THEN e'\\x04f\\x13\\'g\".':::STRING ELSE e'u,t;\\'':::STRING END) STORED, col134_144 STRING AS (CASE WHEN col134_139 IS NULL THEN e'\\x11$}':::STRING ELSE NULL END) STORED, INDEX (col134_143 DESC), UNIQUE (col134_141 ASC), FAMILY (col134_144, col134_139, col134_140), FAMILY (col134_135, col134_141, col134_136), FAMILY (col134_143), FAMILY (col134_138, col134_137))",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-west1\""
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: region \"us-west1\" has not been added to the database (SQLSTATE 42602)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 12a5f45bfd1c988dbbc11b33ec5ce3062f63d948:

          | I211120 12:46:19.543184 233 workload/pgx_helpers.go:72  [-] 68  pgx logger [error]: Exec logParams=map[args:[] err:ERROR: cannot alter a table's LOCALITY if its database is not multi-region enabled (SQLSTATE 42P16) sql:ALTER TABLE schema9.table102 SET LOCALITY REGIONAL BY ROW]
          | I211120 12:46:19.552553 204 workload/pgx_helpers.go:72  [-] 69  pgx logger [error]: Exec logParams=map[args:[] err:ERROR: relation "schema9.table135" does not exist (SQLSTATE 42P01) sql:ALTER TABLE schema9.table135 ALTER COLUMN IrrelevantColumnName DROP STORED]
          | Error: ***FAIL; Failed to receive an execution error when errors were expected
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ``````
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 --txn-log /mnt/data1/cockroach/transactions.json
          |   | ``````
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | DF4C1C8730C0E28FED04100F2EF753A197141806EE26C1015AB4120FB4A7E016CC2411C1142B8A69D00C298F1FC176816FF416C3C1691FC3CFE41B6924C0AD5CEF7C12A925045A943EAC129585D4660C0F2C10AEBF1760C39F0C102343D41285EFCC1C648F619232CEAC10105000040090000000102000040070000001E8204FF067DF341AE4B733D3D36EFC1DA7BF08061B9F8418CD9BA7ADDA3FC41F6AD951A887CF7C174E000A16D9DF1C124E75F0A9234FA41F69E0AB6B3D5EBC142E1E64D014AE8C1A40A664DA77FFE417E85C90CE70DEEC11FC621DFE10BF5C19A4031665264F94160F1623BB3C5CB41AF6BBC7B7101FAC13881F55D01ABE2C1A48E78ECBD760042B1B64875198602C2E74E43B4D2C200C2C000BC414253FC41A08F6568706FD74101020000400400000020082A34D1D4FCC148FE6FAB702DDDC1AEEC3DD18D73E9C17818555819FDD3C119297D0DFD81F1C1B85873FC6E16D0C1A0D8516109E0DF417499C652F9E7F7C1ACFE3650C4E9E341342227EE8EDCED4180151CBDF4FEA2C172FF417C8E22E9C1010200004002000000E458379037CCF041F829BF8DA0F4E341547EE8F463C3DCC13D14B48C0C57F5C1ACECF31B4EAEE3411875D9180028EB41010200004009000000A0DC44589870CAC1E0669EEF1BE7D6411C4042E6B322FE41C05B0CA19796A7413E78D1E91C01F7C14C4D9D182B0BEB41EE0E9952E3FBF84150CC215BB824CBC1B483BD8F8264E941BEA2BAD02886004254AD4CA6956CFB41E894BCFC861FED415C6BBF22392AF941221E940547C6F94187276F2C655FF5C1C410FF7132DBFF4186B6C9E832AA00426C039688E487EBC16001396C105BEA411E94F7E90349F841A4348141EC68FB41A4420194090CE841C0BB80D7718DF64130860083878DD041FEE511385ADBFAC1EC74CBE887AAFF41B5FC959DD11502C2010200004005000000F444C1D4430E01C288FE6CE740C7D7C19C31FB755256E241CE3BFFE802B7EFC1987358F380AFE2C190BB31856FC5F4C1B85A87E0ED10E741DCC9AC56885CD7C1A09A668FBA04DCC108C5C879A798D641C091C19E76F8C8C1148D90CFA25EE6C108EBDE5620B8DD419071B195B632CD4164D5F8B83E08D7C10102000040030000000EEA99D63A65F4C19DBD236193CD00C259BE118081D601C2D8A7AF5916A3D94177805668743DF9C1946D83B3072302C2008F994B0A2391C15051EF1E62AFC741289223BD9BD2F24101020000400600000064A2B0D91D58F8C13AD214E526C3FBC180BE9E7BBAD39F4110619B97795CC241EAC229B3A952E6C152030F2A1A60FA4128BAA49F3FD3D341A4F52AC3B5E5E6C12480368327C2FB41D4EB0A5F5F2FF441E280D8D822E4E4C188762E25432C004274596F3FF5CEFA41F67BFFEEAE2F01422E6CA7120199024280E605E92D15CFC1025157C43C66F941B22D3C18E2F6F5C10102000040070000009186B83925C1F4C1F0009C454F80DDC1F49AD079DC4AE441C08EA78B9876EF413E2FB048F861F6C18E75C7832A2A004226477C2ABA27F2414BE208327F0EFEC1D80E5C532B4DF9C17CAAD5CEA082FF4166FD367C2272F5C10087803D0ABFA641D808CFD1B2B4F641E8B760DB7805D7412CDF3A4EF4E5F4C18294D057DA20F841228CBA1CBD02F2418009268E2530954140D55042C0A8F14110E85BD0DA17F941B4755AAF68EAF94101020000400200000010652773345BD841E0A1CC9CD113D6C1C022A2A14C15FA412109C03DA3D6FFC1A2E376A80B3DF34110A80E2B1E24FEC1':::GEOMETRY,'010700000000000000':::GEOMETRY,'0102000040030000008045BC5F7BF2FD41D2B985D7C796F0C1DCA41E4FFC52F3417A99AB2458CAF041C061029F8DD6C9C1807E77E85C949CC1AA2C2F20985EF2419073EF358417CB41B8BE7825F5A8FF41':::GEOMETRY]",
          |   "ALTER TABLE schema9.table102 SET LOCALITY REGIONAL BY ROW"
          |  ],
          |  "expectedExecErrors": "42P16",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot alter a table's LOCALITY if its database is not multi-region enabled (SQLSTATE 42P16)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:46:18.99632",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TYPE public.enum122 AS ENUM ('d', 'ffsd', 'ssfdsa', 'sssasa', 'faf', 'dda')",
          |   "CREATE SEQUENCE IF NOT EXISTS schema9.seq124",
          |   "ALTER TABLE schema9.table135 ALTER COLUMN IrrelevantColumnName DROP STORED"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema9.table135\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:46:19.025371",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-west1\"",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER DATABASE schemachange ADD REGION \"us-east1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "***FAIL; Failed to receive an execution error when errors were expected"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 67ef976cb1ca1ca306669dff103a218eab4832f3:

          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=4e67030d key=/Table/SystemConfigSpan/Start pri=0.02549104 epo=0 ts=1637844369.165035304,1 min=1637844368.864560334,0 seq=15} lock=true stat=PENDING rts=1637844368.864560334,0 wto=false gul=1637844369.364560334,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:46:08.948061",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TYPE public.enum254 AS ENUM ('ff', 'addass')",
          |   "ALTER DATABASE schemachange ADD REGION \"us-east1\""
          |  ],
          |  "expectedExecErrors": "42710",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: region \"us-east1\" already added to database (SQLSTATE 42710)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:46:08.472419",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE schema63.table277 DROP CONSTRAINT IrrelevantConstraintName"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema63.table277\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:46:08.999304",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange SURVIVE REGION FAILURE"
          |  ],
          |  "expectedExecErrors": "22023",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: at least 3 regions are required for surviving a region failure (SQLSTATE 22023)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:46:08.61983",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE public.table261 (col261_262 REGPROC, col261_263 STRING NULL, col261_264 TIMETZ NULL, col261_265 INTERVAL NULL, col261_266 INT8 NULL, col261_267 \"char\" NULL, col261_268 REGCLASS NOT NULL, col261_269 OID NOT NULL, col261_270 FLOAT4, col261_271 OID NULL, col261_272 TIMESTAMP, col261_273 INT2, PRIMARY KEY (col261_268 DESC), col261_274 STRING NULL AS (lower(CAST(col261_265 AS STRING))) STORED, col261_275 STRING NULL AS (lower(col261_263)) VIRTUAL, INDEX (col261_275 DESC, col261_265 ASC, col261_264 DESC, col261_272, col261_263, (col261_266 + col261_273), col261_271, col261_268 DESC, col261_266, col261_270, col261_269 DESC, col261_273, col261_274 DESC) STORING (col261_262), UNIQUE (col261_267, col261_269 ASC, col261_273 DESC, col261_268, col261_266 DESC, col261_272, col261_262 DESC, col261_271, col261_264 ASC, col261_263 ASC, lower(col261_263), col261_274 DESC), UNIQUE (col261_273 ASC, col261_274 ASC, col261_265 ASC, col261_263, col261_267 DESC, col261_271, col261_266 ASC, col261_275 DESC) STORING (col261_262, col261_264, col261_269, col261_270), INDEX (col261_272, col261_265 DESC, col261_267 DESC, col261_264, col261_263 ASC, col261_274, lower(CAST(col261_262 AS STRING)) DESC, (col261_266 + col261_273) DESC), INDEX (col261_272 DESC, col261_268 ASC, col261_269 DESC, col261_267 DESC, col261_274, lower(CAST(col261_265 AS STRING)) DESC, col261_264, col261_265 DESC, col261_263, col261_270, col261_275 ASC), FAMILY (col261_271, col261_273, col261_272, col261_265, col261_263), FAMILY (col261_266), FAMILY (col261_264, col261_267), FAMILY (col261_274), FAMILY (col261_268, col261_270), FAMILY (col261_269), FAMILY (col261_262))",
          |   "CREATE SEQUENCE IF NOT EXISTS schema63.seq278"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: scanBool: \"\\n\\t\\tSELECT json_array_length(\\n        crdb_internal.pb_to_json(\\n            'cockroach.sql.sqlbase.Descriptor',\\n            descriptor\\n        )-\u003e'table'-\u003e'mutations'\\n       )\\n       \u003e 0\\n\\t\\tFROM system.descriptor\\n\\t  WHERE id = $1::REGCLASS\\n\\t\\t\" [\"public.table261\"]: can't scan into dest[0]: cannot assign NULL to *bool"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 316301aed426e90608e93abd6d33bc53c9069c10:

          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA IF NOT EXISTS schema98 AUTHORIZATION root",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-west1\"",
          |   "CREATE INDEX \"IrrelevantName\" ON schema1.table141 (\"IrrelevantColumn\" ASC)"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema1.table141\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:37:14.29441",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA IF NOT EXISTS schema103 AUTHORIZATION root",
          |   "CREATE TYPE public.enum142 AS ENUM ('addsa', 'sda', 'dda', 'ff', 'afff', 'fdfsd')",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=b141f289 key=/Table/SystemConfigSpan/Start pri=0.00091731 epo=0 ts=1637930234.973063338,1 min=1637930234.294356767,0 seq=24} lock=true stat=PENDING rts=1637930234.294356767,0 wto=false gul=1637930234.794356767,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:37:14.326968",
          |  "ops": [
          |   "BEGIN",
          |   "DROP SCHEMA \"schema1\" CASCADE",
          |   "CREATE TABLE public.table143 (col143_144 DATE NOT NULL, col143_145 INTERVAL NOT NULL, col143_146 REGROLE NOT NULL, col143_147 INET NOT NULL, col143_148 INT8 NULL, col143_149 INT8 NOT NULL, col143_150 REGPROCEDURE NOT NULL, col143_151 STRING NOT NULL, PRIMARY KEY (col143_145 DESC, col143_144 DESC, col143_150 DESC, col143_147, col143_146 ASC), col143_152 INT8 NULL AS (col143_149 + col143_148) VIRTUAL, col143_153 INT8 NULL AS (col143_148 + col143_149) VIRTUAL, col143_154 INT8 NULL AS (col143_149 + col143_148) VIRTUAL, col143_155 STRING NOT NULL AS (lower(col143_151)) STORED, UNIQUE (col143_145 ASC, col143_154 ASC, col143_155 DESC, lower(CAST(col143_150 AS STRING)) ASC, lower(CAST(col143_147 AS STRING)), col143_146, col143_147 DESC, col143_149 ASC) STORING (col143_148, col143_151), FAMILY (col143_151, col143_150, col143_148, col143_147), FAMILY (col143_146, col143_145), FAMILY (col143_149, col143_155, col143_144))",
          |   "ALTER TABLE public.table156 RENAME COLUMN \"IrrelevantColumnName\" TO \"OtherIrrelevantName\""
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.table156\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:37:14.216847",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"europe-west2\"",
          |   "ALTER DATABASE schemachange ADD REGION \"us-west1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "***FAIL; Failed to receive an execution error when errors were expected"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ a998d665653cfd1407a2ca18d53168548c6acb50:

          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:58:50.167979",
          |  "ops": [
          |   "BEGIN",
          |   "DROP SEQUENCE public.seq47"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.seq47\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:58:50.08079",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table48 RENAME COLUMN \"IrrelevantColumnName\" TO \"OtherIrrelevantName\""
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.table48\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:58:50.130929",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange ADD REGION \"us-west1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot add region \"us-west1\" to database schemachange (SQLSTATE 42P12)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:58:49.875662",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq6",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "ALTER SEQUENCE public.seq6 RENAME TO public.seq46",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"europe-west2\""
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: region \"europe-west2\" has not been added to the database (SQLSTATE 42602)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 1d45a24c33db559f70d01ac786bbf5778cfaf7e7:

          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER DATABASE schemachange SURVIVE ZONE FAILURE"
          |  ],
          |  "expectedExecErrors": "42602",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: database must have associated regions before a survival goal can be set (SQLSTATE 42602)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:53:48.717734",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TYPE public.enum17 AS ENUM ('dds', 's', 'dfdds', 'asfdf')",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "CREATE TYPE public.enum49 AS ENUM ('fffa', 'affads', 'addf', 'ddddd')",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=fbe27bb4 key=/Table/SystemConfigSpan/Start pri=0.00370909 epo=0 ts=1638276828.718030192,1 min=1638276828.717670931,0 seq=18} lock=true stat=PENDING rts=1638276828.717670931,0 wto=false gul=1638276829.217670931,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:53:48.716483",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TYPE public.enum7 AS ENUM ('faa', 'dd')",
          |   "CREATE TABLE public.table29 (col29_30 STRING NULL, col29_31 DECIMAL NOT NULL, col29_32 OID[] NULL, col29_33 UUID NULL, col29_34 UUID NULL, col29_35 REGNAMESPACE NULL, col29_36 JSONB, col29_37 REGPROCEDURE NULL, col29_38 STRING NULL, col29_39 TIMETZ[] NULL, col29_40 DATE NOT NULL, col29_41 STRING NULL AS (lower(CAST(col29_33 AS STRING))) STORED, col29_42 STRING NULL AS (lower(CAST(col29_35 AS STRING))) STORED, col29_43 STRING AS (lower(CAST(col29_36 AS STRING))) VIRTUAL, col29_44 STRING AS (lower(CAST(col29_36 AS STRING))) VIRTUAL, col29_45 STRING NULL AS (CASE WHEN col29_39 IS NULL THEN e'\\'':::STRING ELSE e'\\b':::STRING END) VIRTUAL, col29_46 STRING NULL AS (CASE WHEN col29_39 IS NULL THEN e'\\x1cO':::STRING ELSE e'\\U00002603':::STRING END) STORED, col29_47 STRING NULL AS (lower(col29_38)) STORED)",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=3c061beb key=/Table/SystemConfigSpan/Start pri=0.02363566 epo=0 ts=1638276828.718030192,1 min=1638276828.716386056,0 seq=16} lock=true stat=PENDING rts=1638276828.716386056,0 wto=false gul=1638276829.216386056,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:53:48.716011",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA schema5 AUTHORIZATION root",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-west1\"",
          |   "ALTER DATABASE schemachange ADD REGION \"us-east1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "***FAIL; Failed to receive an execution error when errors were expected"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 77f4777848f993b4ded5cc3a026612598ec68947:

          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_PUSHER_ABORTED): \"sql txn\" meta={id=104f1fb3 key=/Table/SystemConfigSpan/Start pri=0.00016573 epo=0 ts=1638363254.461942652,0 min=1638363254.461942652,0 seq=15} lock=true stat=ABORTED rts=1638363254.461942652,0 wto=false gul=1638363254.961942652,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:54:14.261844",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE IF NOT EXISTS schema5.table160 (col160_161 BOOL NOT NULL, col160_162 DECIMAL NOT NULL, col160_163 \"char\" NOT NULL, col160_164 DECIMAL NOT NULL, col160_165 TIMESTAMP NOT NULL, col160_166 FLOAT4 NOT NULL, col160_167 FLOAT4 NOT NULL, PRIMARY KEY (col160_166 DESC, col160_164 ASC, col160_161 DESC, col160_165 ASC, col160_163, col160_167 DESC, col160_162), col160_168 FLOAT4 NOT NULL AS (col160_166 + col160_167) VIRTUAL, col160_169 FLOAT4 NOT NULL AS (col160_166 + 1.12433922290802:::FLOAT8) STORED, col160_170 FLOAT4 NOT NULL AS (col160_166 + col160_167) VIRTUAL, UNIQUE (col160_165, col160_163 ASC, col160_161 ASC, (col160_166 + col160_167) DESC, col160_162, (col160_166 + col160_167) DESC, col160_164, col160_170))",
          |   "ALTER DATABASE schemachange ADD REGION \"us-west1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot add region \"us-west1\" to database schemachange (SQLSTATE 42P12)"
          | }
          |     3.0s        0           16.0           25.7     10.5    109.1    872.4    872.4 opOk
          |     3.0s        0            2.0            2.3    805.3    973.1    973.1    973.1 txnCmtErr
          |     3.0s        0            1.0            2.0    838.9    838.9    838.9    838.9 txnOk
          |     3.0s        0           10.0           24.3      1.1     56.6     56.6     56.6 txnRbk
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError

    cluster.go:1249,context.go:89,cluster.go:1237,test_runner.go:866: dead node detection: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod monitor teamcity-3824394-1638342784-73-n3cpu4-geo --oneshot --ignore-empty-nodes: exit status 1 1: dead (exit status 134)
        2: 9974
        3: 10108
        Error: UNCLASSIFIED_PROBLEM: 1: dead (exit status 134)
        (1) UNCLASSIFIED_PROBLEM
        Wraps: (2) attached stack trace
          -- stack trace:
          | github.com/cockroachdb/cockroach/pkg/roachprod.Monitor
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/roachprod/roachprod.go:596
          | main.glob..func14
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:569
          | main.wrap.func1
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:123
          | github.com/spf13/cobra.(*Command).execute
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:856
          | github.com/spf13/cobra.(*Command).ExecuteC
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:960
          | github.com/spf13/cobra.(*Command).Execute
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:897
          | main.main
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1170
          | runtime.main
          |     /usr/local/go/src/runtime/proc.go:255
          | runtime.goexit
          |     /usr/local/go/src/runtime/asm_amd64.s:1581
        Wraps: (3) 1: dead (exit status 134)
        Error types: (1) errors.Unclassified (2) *withstack.withStack (3) *errutil.leafError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 66a451b4e8e948925782c3877f49d0777c9db812:

          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: column \"crdb_internal_idx_expr_2\" is not a stored computed column (SQLSTATE 42611)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:37:19.494331",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA schema416 AUTHORIZATION root",
          |   "ALTER TABLE public.table420 ALTER COLUMN col420_442 SET DEFAULT \"IrrelevantValue\""
          |  ],
          |  "expectedExecErrors": "42703",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: column \"col420_442\" does not exist (SQLSTATE 42703)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:37:19.429536",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq417",
          |   "ALTER TABLE public.table420 DROP CONSTRAINT \"table420_expr_col420_426_key\""
          |  ],
          |  "expectedExecErrors": "0A000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: unimplemented: cannot drop UNIQUE constraint \"table420_expr_col420_426_key\" using ALTER TABLE DROP CONSTRAINT, use DROP INDEX CASCADE instead (SQLSTATE 0A000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:37:19.722482",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange ADD REGION \"us-west1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot add region \"us-west1\" to database schemachange (SQLSTATE 42P12)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:37:19.415563",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE INDEX index420_444 ON public.table420 (col420_426 ASC, crdb_internal_idx_expr_1 ASC, col420_424 ASC, col420_430, crdb_internal_idx_expr ASC, col420_423 ASC, crdb_internal_idx_expr_2) STORING (col420_429)"
          |  ],
          |  "expectedExecErrors": "XXUUU",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: column \"crdb_internal_idx_expr_1\" is inaccessible and cannot be referenced (SQLSTATE 42703)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ 8fdbf4dacfdf3c2032937fe5359255f3dc13c61a:

          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.seq90\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "13:01:48.197671",
          |  "ops": [
          |   "BEGIN",
          |   "DROP SCHEMA \"public\" CASCADE"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot drop schema \"public\" (SQLSTATE 3F000)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "13:01:47.630022",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects', crdb_internal.validate_multi_region_zone_configs()",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"europe-west2\""
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): \"sql txn\" meta={id=ad0a109b key=/Table/SystemConfigSpan/Start pri=0.05108267 epo=0 ts=1638536508.487357064,1 min=1638536507.629936221,0 seq=8} lock=true stat=PENDING rts=1638536507.629936221,0 wto=false gul=1638536508.129936221,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "13:01:48.063012",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER DATABASE schemachange SURVIVE REGION FAILURE"
          |  ],
          |  "expectedExecErrors": "22023",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: at least 3 regions are required for surviving a region failure (SQLSTATE 22023)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "13:01:48.07254",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE public.table85 (col85_86 GEOGRAPHY NOT NULL, col85_87 REGCLASS, col85_88 REGPROC NULL, col85_89 TIME NOT NULL, INVERTED INDEX (col85_88, col85_89, col85_86 DESC), UNIQUE (col85_89, col85_88, col85_87 ASC), INVERTED INDEX (col85_88, col85_89, col85_86))",
          |   "DROP TABLE public.table85 RESTRICT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: randColumnWithMeta: \"\\n SELECT column_name, data_type, is_nullable\\n   FROM [SHOW COLUMNS FROM public.table85]\\n  WHERE column_name != 'rowid'\\nORDER BY random()\\n  LIMIT 1;\\n\": ERROR: relation \"public.table85\" does not exist (SQLSTATE 42P01)"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

cockroach-teamcity commented 2 years ago

roachtest.schemachange/random-load failed with artifacts on release-21.2 @ de4cc9bf7f6719ef198958de984ab861ca02a8c5:

          |   "CREATE SCHEMA schema22 AUTHORIZATION root",
          |   "ALTER DATABASE schemachange ADD REGION \"us-west1\""
          |  ],
          |  "expectedExecErrors": "42P12",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot add region \"us-west1\" to database schemachange (SQLSTATE 42P12)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:38:20.095632",
          |  "ops": [
          |   "BEGIN",
          |   "DROP VIEW IF EXISTS public.view16 CASCADE",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"europe-west2\"",
          |   "ALTER TABLE public.table39 RENAME TO public.table40"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.table39\" does not exist (SQLSTATE 42P01)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:38:20.2766",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TYPE public.enum27 AS ENUM ('ddafs', 'ads', 'aaa')",
          |   "CREATE TABLE IF NOT EXISTS public.table41 (col41_42 STRING NOT NULL, col41_43 INTERVAL NOT NULL, col41_44 FLOAT4 NOT NULL, col41_45 REGNAMESPACE NOT NULL, col41_46 UUID NOT NULL, col41_47 STRING NOT NULL, col41_48 REGCLASS NOT NULL, col41_49 VARCHAR NOT NULL, col41_50 INT8 NULL, col41_51 GEOGRAPHY, col41_52 BOOL, PRIMARY KEY (col41_46, col41_43 ASC, col41_45), INDEX (lower(col41_47) ASC, col41_48 ASC, col41_50 DESC, col41_46) STORING (col41_49, col41_51))",
          |   "CREATE SCHEMA IF NOT EXISTS schema53 AUTHORIZATION root",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=5ab5ee68 key=/Table/SystemConfigSpan/Start pri=0.02803644 epo=0 ts=1638707900.474718562,1 min=1638707900.276470942,0 seq=31} lock=true stat=PENDING rts=1638707900.276470942,0 wto=false gul=1638707900.776470942,0 (SQLSTATE 40001)"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:38:20.223311",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq37",
          |   "CREATE TYPE public.enum38 AS ENUM ('afa', 'dssa', 'ad')",
          |   "ALTER DATABASE schemachange PRIMARY REGION \"us-east1\"",
          |   "CREATE SCHEMA schema54 AUTHORIZATION root"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: Validation FAIL:\nid 70, db schemachange, schema public, name seq37: relation \"seq37\" (70): invalid locality config: database schemachange is multi-region enabled, but table seq37 has no locality set"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *cluster.WithCommandDetails (4) *exec.ExitError
Reproduce

See: [roachtest README](https://github.com/cockroachdb/cockroach/blob/master/pkg/cmd/roachtest/README.md)

Same failure on other branches

- #63518 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-63484] - #61698 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-release-21.1] - #56235 roachtest: schemachange/random-load failed [C-test-failure O-roachtest O-robot T-sql-schema branch-master]

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!