search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.84k stars 3.77k forks source link

roachtest: schemachange/random-load failed #56235

Closed cockroach-teamcity closed 2 years ago

cockroach-teamcity commented 3 years ago

(jayshrivastava) This issue is resolved by https://github.com/cockroachdb/cockroach/pull/56589, but there are different errors listed in the comments of this issue.

(roachtest).schemachange/random-load failed on master@e25ee7ad4bcdeda49f40eb52c24d2d1b1aa67a86:

          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | ABLE public.table199 RENAME COLUMN "col199_210" TO "col199_211";
          | ROLLBACK; ERROR: column "col199_210" does not exist (SQLSTATE 42703)
          | BEGIN
          |   SELECT 'validating all objects';
          |   ALTER TABLE schema1.table146 ADD COLUMN col146_184 FLOAT8 NULL;
          |   CREATE TABLE schema79.table213 AS SELECT col146_9, col146_1, col146_7, col146_8, col146_2, col146_6, col146_10, col146_4, col146_0, col146_3, col146_5, col146_9, col146_3, col146_2, col146_10, col146_5, col146_7, col146_6, col146_11, col146_4, col146_8, col146_5, col146_0, col146_4, col146_1, col146_11, col146_9, col146_10, col146_3, col146_2, col146_6, col146_7 FROM schema1.table146, schema1.table146, schema1.table146;
          | ROLLBACK; expected SQLSTATE(S) 42712, and got ERROR: source name "table146" specified more than once (missing AS clause) (SQLSTATE 42712)
          | BEGIN
          |   ALTER TABLE schema1.table146 RENAME TO schema1.table186;
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=0ed8e02c key=/Table/SystemConfigSpan/Start pri=0.01781415 epo=0 ts=1604404353.536831218,1 min=1604404352.697989400,0 seq=12} lock=true stat=PENDING rts=1604404352.697989400,0 wto=false max=1604404353.197989400,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          |   SELECT 'validating all objects';
          |   SELECT 'validating all objects';
          |   SELECT 'validating all objects';
          |   ALTER TABLE schema1.table146 ADD COLUMN col146_189 BIT(22) NOT NULL;
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=cd109eac key=/Table/SystemConfigSpan/Start pri=0.00741524 epo=0 ts=1604404353.536831218,1 min=1604404352.560522852,0 seq=11} lock=true stat=PENDING rts=1604404352.560522852,0 wto=false max=1604404353.060522852,0 (SQLSTATE 40001)
          | BEGIN
          |   DROP INDEX schema1.table146@"table146_col146_0_col146_3_col146_10_col146_2_col146_8_col146_11_idx";
          |   ALTER TABLE schema1.table146 RENAME TO schema1.table216;
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=8f983fef key=/Table/SystemConfigSpan/Start pri=0.01290590 epo=0 ts=1604404353.536831218,1 min=1604404353.118079344,0 seq=21} lock=true stat=PENDING rts=1604404353.118079344,0 wto=false max=1604404353.618079344,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER TABLE schema1.table146 RENAME TO schema1.table197;
          |   ALTER VIEW public.view217 RENAME TO public.view218;
          | ROLLBACK; ERROR: relation "public.view217" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   ALTER TABLE schema1.table146 ALTER COLUMN "col146_0" SET NOT NULL;
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=e077351a key=/Table/SystemConfigSpan/Start pri=0.08053295 epo=0 ts=1604404353.794390207,1 min=1604404353.210289453,0 seq=11} lock=true stat=PENDING rts=1604404353.210289453,0 wto=false max=1604404353.710289453,0 (SQLSTATE 40001)
          | BEGIN
          |   DROP TABLE schema1.table146;
          |   SELECT 'validating all objects';
          |   SELECT 'validating all objects';
          |   CREATE SEQUENCE "seq219";
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=c4f974d8 key=/Table/SystemConfigSpan/Start pri=0.00910610 epo=0 ts=1604404353.794390207,1 min=1604404353.302568594,0 seq=19} lock=true stat=PENDING rts=1604404353.302568594,0 wto=false max=1604404353.802568594,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER TABLE schema1.table146 RENAME COLUMN "col146_9" TO "col146_200";
          |   ALTER TABLE schema1.table146 ALTER COLUMN "col146_4" SET NOT NULL;
          |   DROP INDEX schema1.table146@"table146_col146_0_idx";
          |   DROP SEQUENCE "seq131";
          |   DROP TABLE schema1.table146;
          | ROLLBACK; ERROR: cannot update progress on succeeded job (id 604025498857832449) (SQLSTATE XXUUU)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2412951&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

Jira issue: CRDB-2963

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56589

(roachtest).schemachange/random-load failed on master@25edeb8d6c2cefca0c206f86553bc75569f0f2d9:

          |   DROP SCHEMA "schema147" CASCADE;
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   DROP TABLE public.table85;
          |   SELECT 'validating all objects';
          |   CREATE SCHEMA IF NOT EXISTS schema146 AUTHORIZATION root;
          |   CREATE SEQUENCE "seq149";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   SELECT 'validating all objects';
          | COMMIT;  
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE TABLE schema81.table153 AS SELECT "IrrelevantColumnName" FROM schema144.table145;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "schema144.table145" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   DROP INDEX public.table85@"table85_col85_9_col85_0_col85_18_col85_15_col85_6_col85_10_col85_14_col85_13_col85_17_col85_5_col85_8_col85_16_col85_7_key";
          | ROLLBACK; ERROR: index "table85_col85_9_col85_0_col85_18_col85_15_col85_6_col85_10_col85_14_col85_13_col85_17_col85_5_col85_8_col85_16_col85_7_key" is in use as unique constraint (SQLSTATE 2BP01)
          | BEGIN
          |   SELECT 'validating all objects';
          |   ALTER TABLE public.table85 DROP CONSTRAINT "table85_col85_14_col85_11_col85_9_col85_13_col85_6_col85_3_col85_16_col85_17_col85_5_col85_10_col85_4_col85_15_col85_1_key";
          | ROLLBACK; ERROR: unimplemented: cannot drop UNIQUE constraint "table85_col85_14_col85_11_col85_9_col85_13_col85_6_col85_3_col85_16_col85_17_col85_5_col85_10_col85_4_col85_15_col85_1_key" using ALTER TABLE DROP CONSTRAINT, use DROP INDEX CASCADE instead (SQLSTATE 0A000)
          | BEGIN
          |   CREATE VIEW schema81.view156 AS SELECT col85_16, col85_10, col85_14, col85_11, col85_122, col85_2, col85_9, col85_1, col85_7, col85_8, col85_14 FROM public.table85, public.table85;
          | ROLLBACK; expected SQLSTATE(S) 42712, and got ERROR: source name "table85" specified more than once (missing AS clause) (SQLSTATE 42712)
          | BEGIN
          |   CREATE TYPE "public.enum150" AS ENUM ('d', 'f', 'dfsafa', 'sddsaf');
          |   CREATE SCHEMA schema155 AUTHORIZATION root;
          |   ALTER TABLE public.table85 RENAME COLUMN "col85_9" TO "col85_0";
          | ROLLBACK; ERROR: column "col85_0" of relation "table85" already exists (SQLSTATE 42701)
          | BEGIN
          |   ALTER TABLE public.table85 ALTER COLUMN "col85_16" SET NOT NULL;
          |   CREATE VIEW public.view157 AS SELECT col85_15, col85_2, col85_3, col85_7, col85_11, col85_8, col85_6, col85_0, col85_9, col85_17, col85_7, col85_12, col85_11, col85_18, col85_14, col85_122, col85_15, col85_9, col85_3, col85_4, col85_1, col85_2, col85_0, col85_5, col85_16 FROM public.table85, public.table85;
          | ROLLBACK; expected SQLSTATE(S) 42712, and got ERROR: source name "table85" specified more than once (missing AS clause) (SQLSTATE 42712)
          | BEGIN
          |   ALTER TABLE public.table85 ALTER COLUMN "col85_4" SET NOT NULL;
          |   ALTER TABLE public.table85 ALTER COLUMN "col85_11" SET DATA TYPE varchar;
          | ROLLBACK; ERROR: ALTER COLUMN TYPE from regprocedure to varchar is only supported experimentally (SQLSTATE 0A000)
          | BEGIN
          |   ALTER TABLE public.table85 RENAME COLUMN "col85_4" TO "col85_139";
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=ac6e1545 key=/Table/SystemConfigSpan/Start pri=0.02352162 epo=0 ts=1604490358.601257341,1 min=1604490358.161873410,0 seq=11} lock=true stat=PENDING rts=1604490358.161873410,0 wto=false max=1604490358.661873410,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE UNIQUE INDEX IF NOT EXISTS index85_140 ON public.table85 (col85_122, col85_1 ASC, col85_18 ASC, col85_16, col85_12 DESC, col85_9 ASC, col85_2, col85_10, col85_6 ASC, col85_11 ASC, col85_0) STORING (col85_14, col85_3);
          |   CREATE UNIQUE INDEX IF NOT EXISTS index85_160 ON public.table85 (col85_8 ASC) STORING (col85_1, col85_16, col85_15, col85_18, col85_122, col85_7, col85_9, col85_12, col85_5, col85_0);
          |   DROP TABLE public.table85;
          | ROLLBACK; ERROR: cannot update progress on succeeded job (id 604307319754752001) (SQLSTATE XXUUU)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2416377&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56589

(roachtest).schemachange/random-load failed on master@c456295f7d09fbd319509ae4b08fd63dae248ddf:

          | I201105 11:53:24.938039 1 workload/cli/run.go:387  creating load generator... done (took 86.152155ms)
          | Error: Class XX - Internal Error: ERROR: cannot update progress on succeeded job (id 604591900508979201) (SQLSTATE XXUUU)
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | rrent transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE TYPE "schema20.enum21" AS ENUM ('sds', 'dad');
          |   SELECT 'validating all objects';
          |   SELECT 'validating all objects';
          |   CREATE TYPE "public.enum29" AS ENUM ('df', 's', 'aad', 'fddfs', 'dfda', 'sdssss');
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=5037dcf3 key=/Table/SystemConfigSpan/Start pri=0.01451354 epo=0 ts=1604577204.941116396,1 min=1604577204.940401093,0 seq=18} lock=true stat=PENDING rts=1604577204.940401093,0 wto=false max=1604577205.440401093,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          | COMMIT;  
          | BEGIN
          |   ALTER TABLE public.table23 ALTER COLUMN "col23_31" SET DATA TYPE public.enum32;
          | ROLLBACK; ERROR: column "col23_31" does not exist (SQLSTATE 42703)
          | BEGIN
          |   ALTER TABLE public.table23 RENAME TO public.table30;
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=798f62f3 key=/Table/SystemConfigSpan/Start pri=0.00224360 epo=0 ts=1604577205.294855220,1 min=1604577205.040998317,0 seq=12} lock=true stat=PENDING rts=1604577205.040998317,0 wto=false max=1604577205.540998317,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER TABLE public.table23 DROP COLUMN "col23_6";
          | ROLLBACK; ERROR: column "col23_6" is referenced by the primary key (SQLSTATE 42P10)
          | BEGIN
          |   DROP INDEX public.table23@"table23_col23_9_col23_3_col23_5_col23_1_col23_2_col23_6_col23_8_col23_7_col23_0_col23_4_idx";
          |   ALTER TABLE public.table23 DROP COLUMN "col23_4";
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=969f6d6c key=/Table/SystemConfigSpan/Start pri=0.01253458 epo=0 ts=1604577205.584801041,1 min=1604577205.139381106,0 seq=21} lock=true stat=PENDING rts=1604577205.139381106,0 wto=false max=1604577205.639381106,0 (SQLSTATE 40001)
          | BEGIN
          |   DROP SEQUENCE "seq1";
          |   INSERT INTO public.table23 ("col23_0","col23_1","col23_2","col23_3","col23_4","col23_5","col23_6","col23_7","col23_8","col23_9") VALUES ('1993-01-23':::DATE,false,'\x8d9806b76563f5':::BYTES,(-0.8795044422149658):::FLOAT8,false,e'\x04;':::STRING,'':::STRING:::NAME,B'1101100000100000101011000101110111000',3906970349:::OID,3860340046:::OID),('1979-07-10':::DATE,true,'\x6bd845eaeaa902dc':::BYTES,(-0.14666511118412018):::FLOAT8,true,'MKOz':::STRING,e'Yji\x05u>\rL&':::STRING:::NAME,B'0110110111110000110110101000010010111',3442596875:::OID,2299760321:::OID),('1984-03-08':::DATE,false,'\xebcef17b116865':::BYTES,0.08815176039934158:::FLOAT8,true,e'}v\x18d[.<8':::STRING,e'\x1c\x1259X"?':::STRING:::NAME,B'0111100110011000101100101000000000100',2685169037:::OID,4099917808:::OID),('1970-01-05':::DATE,true,'\x4d84ae68':::BYTES,0.0239026490598917:::FLOAT8,true,e'\x01':::STRING,'':::STRING,B'0111111110101011111011100101010000111',130244049:::OID,135908919:::OID),('1980-04-13':::DATE,false,'\x6c5e918dcb1c00':::BYTES,0.3500064015388489:::FLOAT8,true,e'J|(!n\'':::STRING,e'\x04RmJ\x07\r\x04':::STRING:::NAME,B'0100101101111110010010111110101011100',1997859535:::OID,2473432008:::OID);
          |   CREATE SEQUENCE "seq36";
          |   CREATE TABLE public.table38 AS SELECT col23_9, col23_8, col23_5, col23_3, col23_6, col23_4, col23_1, col23_0, col23_7, col23_2, col23_9, col23_6, col23_5, col23_4, col23_3 FROM public.table23, public.table23;
          | ROLLBACK; expected SQLSTATE(S) 42712, and got ERROR: source name "table23" specified more than once (missing AS clause) (SQLSTATE 42712)
          | BEGIN
          |   DROP INDEX public.table23@"table23_col23_9_col23_3_col23_5_col23_1_col23_2_col23_6_col23_8_col23_7_col23_0_col23_4_idx";
          |   ALTER TABLE public.table23 ALTER COLUMN "col23_9" DROP DEFAULT;
          |   DROP TABLE public.table23;
          | ROLLBACK; ERROR: cannot update progress on succeeded job (id 604591900508979201) (SQLSTATE XXUUU)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2418892&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

jayshrivastava commented 3 years ago

Repros consistently on master:

root@:26257/defaultdb> create table t();
CREATE TABLE

root@:26257/defaultdb> begin;
BEGIN

root@:26257/defaultdb  OPEN> alter table t add column foo int;
ALTER TABLE

root@:26257/defaultdb  OPEN> drop table t;
ERROR: cannot update progress on succeeded job (id 604652781674594305)

The problem seems to be here: https://github.com/cockroachdb/cockroach/blob/53e7cc5ab0d65157dcfb922e8bf719ac8b65d6d5/pkg/sql/table.go#L101

A job gets created for the ALTER TABLE (and presumably succeeds). We call the function again for DROP TABLE, grab the ALTER TABLE job from the cache, and try to update the job that has succeeded. The function errors out here: https://github.com/cockroachdb/cockroach/blob/53e7cc5ab0d65157dcfb922e8bf719ac8b65d6d5/pkg/sql/table.go#L186

@ajwerner @lucy-zhang

thoszhang commented 3 years ago

After the ALTER statement the job should be running. My guess is that what's actually marking the job as succeeded is the DROP TABLE:

https://github.com/cockroachdb/cockroach/blob/53e7cc5ab0d65157dcfb922e8bf719ac8b65d6d5/pkg/sql/drop_table.go#L428-L446

This would interact poorly with the later job update.

thoszhang commented 3 years ago

I think what we want to do here is delete the schema change job that was queued in the same transaction if there was one, in the place linked above where we're currently marking jobs as succeeded. This would be stored in extendedEvalContext.SchemaChangeJobCache. There's not really a point in writing jobs that start off as succeeded. We could also just replace the contents of the job with the new (drop) contents, but that seems pointlessly complicated.

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56184

(roachtest).schemachange/random-load failed on master@37653515b3b6a612b9ee71c4f3bcfb9511805edb:

          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | .table32, public.table7, public.table7;
          | ROLLBACK; expected SQLSTATE(S) 42712,42P01,42P07, and got ERROR: relation "public.table32" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   DROP TABLE public.table45;
          | ROLLBACK; ERROR: relation "public.table45" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   ALTER TABLE public.table7 ALTER COLUMN "col7_48" SET NOT NULL;
          | ROLLBACK; ERROR: column "col7_48" does not exist (SQLSTATE 42703)
          | BEGIN
          |   CREATE SCHEMA schema46 AUTHORIZATION root;
          |   DROP SCHEMA "schema49" CASCADE;
          | ROLLBACK; ERROR: unknown schema "schema49" (SQLSTATE 3F000)
          | BEGIN
          |   INSERT INTO public.table7 ("col7_0","col7_1","col7_2","col7_3","col7_4","col7_5","col7_6","col7_7","col7_8","col7_9","col7_10") VALUES (5.202468463915634480E+30:::DECIMAL,'':::STRING:::NAME,2991302138:::OID,'14:55:23.019024-06:54:00':::TIMETZ,e'A+/\r\x1dM3E':::STRING,'00:23:11.077947':::TIME,B'0100000101001001111011000111001101100000110',2100687817:::OID,true,B'01010110000001101011000111000111010100100010',e't-hjeV\x132':::STRING),(7.977353494038566752E+22:::DECIMAL,'A':::STRING:::NAME,2225078512:::OID,'18:22:48.380329+07:52:00':::TIMETZ,'-cRN=':::STRING,'19:12:56.896323':::TIME,B'1101000110001000000001110001010101111101011',912509751:::OID,true,B'00011101011100111110001100111111101001100000','+':::STRING);
          |   ALTER TABLE public.table7 DROP CONSTRAINT "table7_col7_2_col7_3_col7_10_col7_0_col7_6_col7_5_col7_8_col7_4_col7_9_col7_1_key";
          | ROLLBACK; ERROR: unimplemented: schema change statement cannot follow a statement that has written in the same transaction (SQLSTATE 0A000)
          | BEGIN
          |   CREATE TABLE public.table37 AS SELECT col7_9, col7_6, col7_3, col7_2, col7_10, col7_8, col7_5, col7_1 FROM public.table7;
          | COMMIT;  
          | BEGIN
          |   ALTER TABLE public.table7 DROP COLUMN "col7_3";
          |   CREATE INVERTED INDEX index7_51 ON public.table7 (col7_2, col7_6 ASC, col7_0 ASC, col7_10, col7_9 ASC, col7_1 ASC, col7_7 DESC, col7_8 DESC) STORING (col7_4);
          | ROLLBACK; ERROR: inverted indexes don't support stored columns (SQLSTATE 26000)
          | BEGIN
          |   CREATE SEQUENCE "seq54";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   CREATE TABLE public.table53 AS SELECT "IrrelevantColumnName" FROM public.table47;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table47" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE INDEX IF NOT EXISTS index7_33 ON public.table7 (col7_1 ASC, col7_5 DESC, col7_0, col7_3 ASC, col7_2, col7_9, col7_4, col7_8 DESC, col7_6 DESC, col7_10 DESC, col7_7);
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=22765b68 key=/Table/SystemConfigSpan/Start pri=0.04133284 epo=0 ts=1604663822.186152765,1 min=1604663821.883143275,0 seq=11} lock=true stat=PENDING rts=1604663821.883143275,0 wto=false max=1604663822.383143275,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER SEQUENCE "seq57" RENAME TO "seq58";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   ALTER TABLE public.table37 RENAME COLUMN "col7_6" TO "col37_59";
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=c8ca71dc key=/Table/SystemConfigSpan/Start pri=0.10351483 epo=0 ts=1604663822.195669475,1 min=1604663822.128909564,0 seq=11} lock=true stat=PENDING rts=1604663822.128909564,0 wto=false max=1604663822.628909564,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER TABLE public.table7 ALTER COLUMN "col7_2" DROP DEFAULT;
          |   CREATE TYPE "public.enum60" AS ENUM ('sadsa', 'fadsa');
          |   DROP SCHEMA "schema62" CASCADE;
          | ROLLBACK; ERROR: unknown schema "schema62" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE VIEW public.view64 AS SELECT col7_10, col7_4, col7_8, col7_7, col7_3, col7_10 FROM public.table7, public.table37;
          | ***FAIL; Expected no errors, but got ERROR: column reference "col7_10" is ambiguous (candidates: public.table7.col7_10, public.table37.col7_10) (SQLSTATE 42702)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2421874&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

jayshrivastava commented 3 years ago

The ambiguous column issue will be fixed in https://github.com/cockroachdb/cockroach/pull/56184

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56379

(roachtest).schemachange/random-load failed on master@61c96aaca632dfba5154d55d82c5af0732053b72:

          | main.registerSchemaChangeRandomLoad.func1
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/schemachange_random_load.go:39
          | main.(*testRunner).runTest.func2
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:755
          | runtime.goexit
          |     /usr/local/go/src/runtime/asm_amd64.s:1357
        Wraps: (2) output in run_111730.847_n1_workload_run_schemachange
        Wraps: (3) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2424980-1604732977-81-n3cpu4:1 -- ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 returned
          | stderr:
          | I201107 11:17:31.660522 1 workload/cli/run.go:356  creating load generator...
          | I201107 11:17:31.734537 1 workload/cli/run.go:387  creating load generator... done (took 74.013971ms)
          | Error: ***UNEXPECTED ERROR: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=d6fdc90c key=/Table/SystemConfigSpan/Start pri=0.00808011 epo=0 ts=1604747857.654492452,1 min=1604747857.129639707,0 seq=6} lock=true stat=PENDING rts=1604747857.129639707,0 wto=false max=1604747857.129639707,0 (SQLSTATE 40001)
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          |  INT4 NOT NULL, col399_6 VARCHAR NULL, col399_7 FLOAT4, col399_8 VARCHAR NOT NULL, col399_9 STRING NOT NULL, col399_10 INT8 NOT NULL, col399_11 REGPROCEDURE NOT NULL, col399_12 FLOAT4 NOT NULL, col399_13 REGNAMESPACE NOT NULL, col399_14 REGNAMESPACE NOT NULL, col399_15 BIT(33) NOT NULL, PRIMARY KEY (col399_9 ASC, col399_14 DESC, col399_4 ASC, col399_13 DESC, col399_15 DESC, col399_11 DESC, col399_12 ASC, col399_3, col399_2 DESC, col399_5 DESC, col399_8 ASC, col399_1 ASC), UNIQUE (col399_15 ASC, col399_12 ASC) STORING (col399_6, col399_7), UNIQUE (col399_8, col399_1 ASC, col399_7 ASC, col399_0 DESC, col399_2, col399_12, col399_9, col399_14 DESC, col399_6 ASC, col399_13 ASC, col399_11 ASC, col399_15 DESC), UNIQUE (col399_6 ASC, col399_14 DESC, col399_0, col399_7 DESC, col399_5, col399_1, col399_3 ASC, col399_13 DESC, col399_4 ASC, col399_9 ASC, col399_8 DESC) STORING (col399_10), UNIQUE (col399_3, col399_14, col399_4 DESC, col399_2 DESC, col399_5, col399_11 DESC, col399_6), INDEX (col399_14, col399_7, col399_6 DESC, col399_15 DESC, col399_12, col399_1 ASC, col399_8, col399_10 DESC, col399_13 ASC, col399_0 ASC, col399_9 DESC, col399_2 DESC, col399_11 DESC), UNIQUE (col399_7, col399_13 ASC, col399_8, col399_1 ASC, col399_3, col399_14 DESC, col399_11, col399_0 ASC, col399_2 DESC, col399_15 DESC, col399_12 ASC, col399_5 DESC, col399_9 DESC, col399_10 ASC, col399_4), INDEX (col399_10 DESC, col399_14 DESC, col399_9 DESC, col399_7 ASC, col399_6 ASC, col399_12 ASC, col399_4 ASC, col399_1 ASC, col399_11 DESC, col399_8 ASC, col399_3 DESC) STORING (col399_0), FAMILY (col399_7, col399_3, col399_0, col399_6, col399_8, col399_12, col399_14), FAMILY (col399_4), FAMILY (col399_5), FAMILY (col399_9, col399_13, col399_1, col399_2, col399_11), FAMILY (col399_10, col399_15));
          |   CREATE SEQUENCE "seq402";
          |   ALTER TABLE public.table395 DROP COLUMN "col360_1";
          | ROLLBACK; ERROR: column "col360_1" is referenced by the primary key (SQLSTATE 42P10)
          | BEGIN
          |   SELECT 'validating all objects';
          |   DROP SEQUENCE "seq407";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   CREATE TABLE public.table405 AS SELECT col360_6, col360_4, col360_6, col360_4, col360_1, col360_8, col360_7, col360_3, col360_0, col360_2, col360_5 FROM public.table395, public.table395;
          | ROLLBACK; expected SQLSTATE(S) 42712, and got ERROR: source name "table395" specified more than once (missing AS clause) (SQLSTATE 42712)
          | BEGIN
          |   ALTER TABLE public.table395 ALTER COLUMN "col360_3" DROP STORED;
          | ROLLBACK; ERROR: column "col360_3" is not a computed column (SQLSTATE 42611)
          | _elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
          |     6.0s        0           44.0           49.0     10.0    637.5    738.2    738.2 opOk
          |     6.0s        0            8.0            8.0    671.1   1006.6   1006.6   1006.6 txnCmtErr
          |     6.0s        0            1.0            3.2    285.2    285.2    285.2    285.2 txnOk
          |     6.0s        0           31.0           27.3      5.5    251.7    268.4    268.4 txnRbk
          | BEGIN
          |   ALTER TABLE public.table360 ALTER COLUMN "col360_7" DROP DEFAULT;
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=6b57befd key=/Table/SystemConfigSpan/Start pri=0.01029316 epo=0 ts=1604747857.539464352,1 min=1604747857.046405729,0 seq=6} lock=true stat=PENDING rts=1604747857.046405729,0 wto=false max=1604747857.046405729,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER TABLE public.table360 ADD COLUMN col360_392 UUID NOT NULL;
          | ***FAIL; Expected no errors, but got ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=d6fdc90c key=/Table/SystemConfigSpan/Start pri=0.00808011 epo=0 ts=1604747857.654492452,1 min=1604747857.129639707,0 seq=6} lock=true stat=PENDING rts=1604747857.129639707,0 wto=false max=1604747857.129639707,0 (SQLSTATE 40001)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2424980&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56184

(roachtest).schemachange/random-load failed on master@be84ae4a59e6a5e50694c1756e05b0ba7c3a54d8:

          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          |
          |   CREATE SCHEMA schema143 AUTHORIZATION root;
          |   DROP TABLE public.table44;
          | ROLLBACK; ERROR: restart transaction: failed to mark job 605437383022870529 as as successful: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=ac574d9d key=/Table/SystemConfigSpan/Start pri=0.00999141 epo=0 ts=1604835227.205064106,1 min=1604835226.261765564,0 seq=27} lock=true stat=PENDING rts=1604835226.261765564,0 wto=false max=1604835226.261765564,0 (SQLSTATE 40001)
          | BEGIN
          |   DROP SEQUENCE "seq130";
          |   ALTER TABLE public.table44 RENAME TO public.table139;
          |   ALTER TABLE public.table139 RENAME COLUMN "col44_4" TO "col139_150";
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=5f3c961c key=/Table/SystemConfigSpan/Start pri=0.00108127 epo=0 ts=1604835227.020530018,1 min=1604835226.302017758,0 seq=31} lock=true stat=PENDING rts=1604835226.302017758,0 wto=false max=1604835226.802017758,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER TABLE public.table44 ALTER COLUMN "col44_5" DROP DEFAULT;
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=c22ccdbf key=/Table/SystemConfigSpan/Start pri=0.04105363 epo=0 ts=1604835227.224508847,1 min=1604835226.430106189,0 seq=11} lock=true stat=PENDING rts=1604835226.430106189,0 wto=false max=1604835226.930106189,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          | COMMIT;  
          | BEGIN
          |   SELECT 'validating all objects';
          | COMMIT;  
          | BEGIN
          |   DROP SEQUENCE "seq130";
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=50a8fd35 key=/Table/SystemConfigSpan/Start pri=0.03107232 epo=0 ts=1604835227.273255375,1 min=1604835226.613504721,0 seq=11} lock=true stat=PENDING rts=1604835226.613504721,0 wto=false max=1604835227.113504721,0 (SQLSTATE 40001)
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   DROP INDEX public.table44@"table44_col44_4_col44_3_col44_1_col44_2_col44_6_col44_0_col44_5_col44_7_key";
          | ROLLBACK; ERROR: index "table44_col44_4_col44_3_col44_1_col44_2_col44_6_col44_0_col44_5_col44_7_key" is in use as unique constraint (SQLSTATE 2BP01)
          | BEGIN
          |   ALTER TABLE public.table44 ALTER COLUMN "col44_4" SET DATA TYPE char;
          | ROLLBACK; ERROR: ALTER COLUMN TYPE from regclass to char is only supported experimentally (SQLSTATE 0A000)
          | BEGIN
          |   ALTER TABLE public.table44 ALTER COLUMN "col44_2" DROP STORED;
          | ROLLBACK; ERROR: column "col44_2" is not a computed column (SQLSTATE 42611)
          | BEGIN
          |   INSERT INTO public.table44 ("col44_0","col44_1","col44_2","col44_3","col44_4","col44_5","col44_118","col44_7","rowid") VALUES (B'1110001111110111000101100110010010','BOX(-0.39836128898639184 -1.9211638265748125,0.6555088803108111 -0.26072180432050374)':::BOX2D,0.3202575230258935:::FLOAT8,2238723926:::OID,2330944541:::OID,B'10','03:42:20.519765-05:11:00':::TIMETZ,'f691:2f66:7303:a77c:4dd7:f57a:6e41:2172/85':::INET,(-1):::INT8);
          |   CREATE TABLE IF NOT EXISTS public.table154 (col154_0 INET, col154_1 REGNAMESPACE NOT NULL, col154_2 FLOAT8, col154_3 REGTYPE, col154_4 REGPROCEDURE, col154_5 REGPROCEDURE NULL, col154_6 "char" NULL, col154_7 FLOAT8, col154_8 OID NULL, col154_9 BOX2D NOT NULL, col154_10 OID NOT NULL, col154_11 DECIMAL NULL, col154_12 BOX2D, UNIQUE (col154_12 DESC, col154_0 DESC, col154_5 ASC) STORING (col154_3, col154_10), UNIQUE (col154_10 ASC, col154_9 DESC, col154_0 DESC, col154_6 DESC, col154_7 ASC, col154_4, col154_3 ASC), UNIQUE (col154_4 DESC, col154_9 ASC, col154_6, col154_8, col154_11 DESC, col154_3), INDEX (col154_0));
          | ***FAIL; Expected no errors, but got ERROR: unimplemented: schema change statement cannot follow a statement that has written in the same transaction (SQLSTATE 0A000)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2426238&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56589

(roachtest).schemachange/random-load failed on master@f2b53d1080f1ccc503263dd178ee31b4878d2360:

          | ROLLBACK; ERROR: column "col127_3" is referenced by the primary key (SQLSTATE 42P10)
          | BEGIN
          |   ALTER TABLE schema3.table127 DROP CONSTRAINT "table127_col127_4_col127_1_col127_2_col127_5_col127_3_key";
          | ROLLBACK; ERROR: unimplemented: cannot drop UNIQUE constraint "table127_col127_4_col127_1_col127_2_col127_5_col127_3_key" using ALTER TABLE DROP CONSTRAINT, use DROP INDEX CASCADE instead (SQLSTATE 0A000)
          | BEGIN
          |   CREATE SEQUENCE "seq92";
          |   ALTER SEQUENCE "seq92" RENAME TO "seq99";
          | COMMIT;  
          | BEGIN
          |   DROP SEQUENCE "seq99";
          |   CREATE TABLE schema3.table127 (col127_0 BIT NULL, col127_1 TIMESTAMP NOT NULL, col127_2 INT4, col127_3 REGNAMESPACE NOT NULL, col127_4 VARCHAR, col127_5 DECIMAL NOT NULL, PRIMARY KEY (col127_3 DESC), INDEX (col127_0 ASC, col127_1), UNIQUE (col127_4, col127_1 ASC, col127_2 DESC, col127_5, col127_3), INDEX (col127_5 DESC, col127_0 ASC, col127_3 ASC, col127_1, col127_4 ASC) STORING (col127_2), INDEX (col127_2 DESC, col127_3 ASC, col127_0, col127_5 ASC) STORING (col127_4), INDEX (col127_2 DESC, col127_5 ASC), INDEX (col127_2 DESC, col127_3 DESC, col127_5, col127_4 ASC), UNIQUE (col127_0 ASC, col127_1 DESC) STORING (col127_4, col127_5), UNIQUE (col127_5 DESC, col127_1 DESC, col127_3 ASC) STORING (col127_0, col127_2), INDEX (col127_3 ASC, col127_4, col127_2, col127_1 DESC), FAMILY (col127_5, col127_0, col127_1, col127_4), FAMILY (col127_2, col127_3));
          |   SELECT 'validating all objects';
          | COMMIT;  
          | BEGIN
          |   SELECT 'validating all objects';
          | COMMIT;  
          | BEGIN
          |   ALTER TABLE schema3.table127 ALTER COLUMN "col127_0" SET DATA TYPE regprocedure;
          | ROLLBACK; ERROR: cannot convert BIT to REGPROCEDURE (SQLSTATE 42846)
          | BEGIN
          |   ALTER TABLE schema3.table127 RENAME CONSTRAINT "table127_col127_5_col127_0_col127_3_col127_1_col127_4_idx" TO "index127_164";
          | ROLLBACK; ERROR: constraint "table127_col127_5_col127_0_col127_3_col127_1_col127_4_idx" does not exist (SQLSTATE 42704)
          | BEGIN
          |   ALTER TABLE schema3.table127 ALTER COLUMN "col127_3" DROP NOT NULL;
          | ROLLBACK; ERROR: column "col127_3" is in a primary index (SQLSTATE 42P16)
          | BEGIN
          |   DROP SCHEMA "schema3" CASCADE;
          |   SELECT 'validating all objects';
          |   CREATE SEQUENCE "seq166";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          |     3.0s        0           37.0           37.7     11.5    192.9    268.4    268.4 opOk
          |     3.0s        0            4.0            6.3    704.6    838.9    838.9    838.9 txnCmtErr
          |     3.0s        0            7.0            3.0    520.1   2080.4   2080.4   2080.4 txnOk
          |     3.0s        0           31.0           22.7      2.5     25.2     71.3     71.3 txnRbk
          | BEGIN
          |   ALTER TABLE schema3.table127 ALTER COLUMN "col127_1" DROP NOT NULL;
          |   CREATE INDEX IF NOT EXISTS index127_170 ON schema3.table127 (col127_0, col127_5 ASC, col127_1 DESC, col127_3 ASC, col127_2 DESC, col127_4 ASC);
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=c9601d07 key=/Table/SystemConfigSpan/Start pri=0.06435304 epo=0 ts=1605008062.011912142,1 min=1605008061.723009715,0 seq=19} lock=true stat=PENDING rts=1605008061.723009715,0 wto=false max=1605008062.223009715,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER TABLE schema3.table127 ALTER COLUMN "col127_3" DROP DEFAULT;
          |   ALTER TABLE schema3.table127 ALTER COLUMN "col127_1" DROP STORED;
          | ROLLBACK; ERROR: column "col127_1" is not a computed column (SQLSTATE 42611)
          | BEGIN
          |   ALTER TABLE schema3.table127 DROP COLUMN "col127_5";
          |   ALTER TABLE schema3.table127 RENAME COLUMN "col127_1" TO "col127_171";
          |   ALTER TABLE schema3.table127 ADD COLUMN col127_172 TIME NOT NULL;
          |   DROP TABLE schema3.table127;
          | ROLLBACK; ERROR: cannot update progress on succeeded job (id 606003730385895425) (SQLSTATE XXUUU)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2430130&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56379

(roachtest).schemachange/random-load failed on master@f2b53d1080f1ccc503263dd178ee31b4878d2360:

          | I201110 20:48:23.825440 1 workload/cli/run.go:356  creating load generator...
          | I201110 20:48:23.938752 1 workload/cli/run.go:387  creating load generator... done (took 113.310449ms)
          | Error: ***UNEXPECTED ERROR: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=2706b962 key=/Table/SystemConfigSpan/Start pri=0.02894662 epo=0 ts=1605041314.759424866,2 min=1605041314.484833825,0 seq=6} lock=true stat=PENDING rts=1605041314.484833825,0 wto=false max=1605041314.484833825,0 (SQLSTATE 40001)
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | : retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=45c15d0d key=/Table/SystemConfigSpan/Start pri=0.00730315 epo=0 ts=1605041314.385718935,1 min=1605041313.656857515,0 seq=12} lock=true stat=PENDING rts=1605041313.656857515,0 wto=false max=1605041314.156857515,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER TABLE schema179.table524 RENAME TO schema179.table555;
          |   ALTER TABLE schema330.table557 ALTER COLUMN "col557_558" DROP DEFAULT;
          | ROLLBACK; ERROR: relation "schema330.table557" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE VIEW schema330.view548 AS SELECT col524_0 FROM schema179.table524;
          |   ALTER TABLE schema179.table524 RENAME TO public.table560;
          | ROLLBACK; ERROR: cannot rename relation "schema179.table524" because view "view548" depends on it (SQLSTATE 2BP01)
          | BEGIN
          |   ALTER TABLE schema179.table524 ADD COLUMN col524_553 BIT(48)[];
          |   CREATE SEQUENCE "seq563";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   DROP SEQUENCE "seq438";
          |   CREATE INVERTED INDEX index524_559 ON schema179.table524 (col524_0);
          |   CREATE TABLE IF NOT EXISTS schema179.table524 (col524_0 BIT(26) NOT NULL, col524_1 INT4 NOT NULL, col524_2 OID NOT NULL, col524_3 VARCHAR NOT NULL, col524_4 INTERVAL NOT NULL, col524_5 BYTES NOT NULL, col524_6 TIME NOT NULL, col524_7 TIME NOT NULL, col524_8 DECIMAL NOT NULL, col524_9 CHAR NOT NULL, col524_10 CHAR NOT NULL, col524_11 CHAR NOT NULL, col524_12 REGCLASS NOT NULL, col524_13 INT4 NOT NULL, col524_14 REGTYPE NOT NULL, col524_15 FLOAT8 NOT NULL, col524_16 INTERVAL NOT NULL, col524_17 REGCLASS NOT NULL, PRIMARY KEY (col524_0 DESC, col524_5 ASC, col524_12 DESC, col524_13 ASC, col524_1 ASC, col524_15 ASC, col524_11, col524_10 ASC, col524_16, col524_14 DESC, col524_4, col524_17 ASC, col524_6, col524_2 ASC, col524_9 ASC, col524_7, col524_3 ASC), INDEX (col524_6, col524_15 ASC, col524_13 DESC, col524_11 ASC, col524_5 ASC, col524_7, col524_14 ASC));
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=e5b24a32 key=/Table/SystemConfigSpan/Start pri=0.01230785 epo=0 ts=1605041314.510029150,1 min=1605041314.385718935,0 seq=22} lock=true stat=PENDING rts=1605041314.385718935,0 wto=false max=1605041314.885718935,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          |   ALTER VIEW schema179.view564 RENAME TO schema179.view565;
          | ROLLBACK; ERROR: relation "schema179.view564" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE VIEW schema569.view570 AS SELECT col524_0 FROM schema179.table524;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot create "schema569.view570" because the target database or schema does not exist (SQLSTATE 3F000)
          | BEGIN
          |   CREATE TABLE schema179.table524 AS SELECT col524_0, "IrrelevantColumnName" FROM schema179.table524, schema179.table568;
          | ROLLBACK; expected SQLSTATE(S) 42P01,42P07, and got ERROR: relation "schema179.table568" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   ALTER TABLE schema179.table524 RENAME COLUMN "col524_0" TO "col524_567";
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=f46a47fb key=/Table/SystemConfigSpan/Start pri=0.00648629 epo=0 ts=1605041314.759424866,2 min=1605041314.432044445,0 seq=6} lock=true stat=PENDING rts=1605041314.432044445,0 wto=false max=1605041314.432044445,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER TABLE schema179.table524 ADD COLUMN col524_566 REGCLASS NULL;
          | ***FAIL; Expected no errors, but got ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=2706b962 key=/Table/SystemConfigSpan/Start pri=0.02894662 epo=0 ts=1605041314.759424866,2 min=1605041314.484833825,0 seq=6} lock=true stat=PENDING rts=1605041314.484833825,0 wto=false max=1605041314.484833825,0 (SQLSTATE 40001)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2431438&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) tracked by #57487

(roachtest).schemachange/random-load failed on master@df21e5065a79d1dd5b0fd76018724dd26660beb2:

          | ROLLBACK; ERROR: relation "public.table95" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   DROP SCHEMA "schema102" CASCADE;
          | ROLLBACK; ERROR: unknown schema "schema102" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE TYPE "public.enum82" AS ENUM ('asds', 'sd', 'da');
          |   CREATE SEQUENCE "seq90";
          |   DROP SEQUENCE "seq90";
          |   CREATE SEQUENCE "seq103";
          |   CREATE SEQUENCE "seq106";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE TABLE IF NOT EXISTS public.table107 (col107_0 INT2, col107_1 INET, col107_2 CHAR, col107_3 REGCLASS, col107_4 OID NULL, INDEX (col107_3 ASC, col107_1 DESC), UNIQUE (col107_0, col107_3 ASC) STORING (col107_1, col107_4), UNIQUE (col107_0 ASC) STORING (col107_1, col107_4), UNIQUE (col107_0, col107_3 ASC) STORING (col107_1), UNIQUE (col107_1 DESC, col107_3 ASC, col107_4 ASC, col107_2 ASC, col107_0));
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=2dd96b17 key=/Table/SystemConfigSpan/Start pri=0.00806446 epo=0 ts=1605094077.057379012,1 min=1605094076.949145118,0 seq=7} lock=true stat=PENDING rts=1605094076.949145118,0 wto=false max=1605094077.449145118,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE SEQUENCE "seq75";
          |   ALTER SEQUENCE "seq75" RENAME TO "seq108";
          |   CREATE SCHEMA schema109 AUTHORIZATION root;
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=30ab56da key=/Table/SystemConfigSpan/Start pri=0.04627681 epo=0 ts=1605094077.748966737,1 min=1605094076.657182731,0 seq=28} lock=true stat=PENDING rts=1605094076.657182731,0 wto=false max=1605094076.657182731,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE TYPE "schema110.enum111" AS ENUM ('afsas', 'daads', 'adsaas', 'dsfss', 'add', 'fsfd');
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE SCHEMA IF NOT EXISTS schema60 AUTHORIZATION root;
          |   SELECT 'validating all objects';
          | COMMIT;  
          | BEGIN
          |   SELECT 'validating all objects';
          | COMMIT;  
          |     2.0s        0           26.9           34.9      7.1    159.4    226.5    226.5 opOk
          |     2.0s        0            2.0            5.0    805.3    838.9    838.9    838.9 txnCmtErr
          |     2.0s        0            3.0            2.5   1140.9   1140.9   1140.9   1140.9 txnOk
          |     2.0s        0           19.9           23.0     10.5    134.2    142.6    142.6 txnRbk
          | BEGIN
          |   CREATE SCHEMA schema113 AUTHORIZATION root;
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=d448ac51 key=/Table/SystemConfigSpan/Start pri=0.00009094 epo=0 ts=1605094077.923063237,1 min=1605094077.774945946,0 seq=6} lock=true stat=PENDING rts=1605094077.774945946,0 wto=false max=1605094077.774945946,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER TABLE public.table112 ALTER COLUMN "col112_119" DROP STORED;
          | ROLLBACK; ERROR: column "col112_119" does not exist (SQLSTATE 42703)
          | BEGIN
          |   CREATE SEQUENCE "seq115";
          | COMMIT;  
          | BEGIN
          |   CREATE TABLE schema60.table98 (col98_0 INT4, col98_1 TIMESTAMP NULL, col98_2 REGTYPE NOT NULL, col98_3 TIMETZ, PRIMARY KEY (col98_2), UNIQUE (col98_1 ASC, col98_2, col98_0 ASC, col98_3 ASC), INDEX (col98_0 ASC, col98_3 ASC, col98_1 DESC), UNIQUE (col98_2), INDEX (col98_1 DESC, col98_0 ASC, col98_3), INDEX (col98_2 DESC, col98_0 DESC, col98_1), UNIQUE (col98_0, col98_1 DESC) STORING (col98_3));
          | ***FAIL; Expected no errors, but got ERROR: descriptor is being dropped (SQLSTATE XXUUU)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2433350&tab=artifacts#/schemachange/random-load) Related: - #56526 roachtest: schemachange/random-load failed [C-test-failure](https://api.github.com/repos/cockroachdb/cockroach/labels/C-test-failure) [O-roachtest](https://api.github.com/repos/cockroachdb/cockroach/labels/O-roachtest) [O-robot](https://api.github.com/repos/cockroachdb/cockroach/labels/O-robot) [branch-56248](https://api.github.com/repos/cockroachdb/cockroach/labels/branch-56248) [release-blocker](https://api.github.com/repos/cockroachdb/cockroach/labels/release-blocker) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56379

(roachtest).schemachange/random-load failed on master@2d34be73030a0d114b84fc2785672eda072c5a50:

          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | 3F000)
          | BEGIN
          |   DROP TABLE public.table137;
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=080adeb4 key=/Table/SystemConfigSpan/Start pri=0.00444837 epo=0 ts=1605267084.253223995,2 min=1605267084.062323607,0 seq=6} lock=true stat=PENDING rts=1605267084.062323607,0 wto=false max=1605267084.062323607,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE TABLE public.table166 AS SELECT col137_2, col137_0, col137_1, col137_2 FROM public.table137, public.table137;
          | ROLLBACK; expected SQLSTATE(S) 42712, and got ERROR: source name "table137" specified more than once (missing AS clause) (SQLSTATE 42712)
          | BEGIN
          |   SELECT 'validating all objects';
          |   ALTER TABLE public.table137 RENAME COLUMN "col137_1" TO "col137_162";
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=e9e3b228 key=/Table/SystemConfigSpan/Start pri=0.02318821 epo=0 ts=1605267084.574052713,1 min=1605267084.206106010,0 seq=6} lock=true stat=PENDING rts=1605267084.206106010,0 wto=false max=1605267084.206106010,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE SEQUENCE "seq160";
          |   ALTER TABLE schema150.table165 ADD COLUMN IrrelevantColumnName string;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "schema150.table165" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   ALTER TABLE public.table161 ALTER COLUMN "col137_2" DROP STORED;
          | ROLLBACK; ERROR: column "col137_2" is not a computed column (SQLSTATE 42611)
          | BEGIN
          |   ALTER TABLE public.table137 RENAME CONSTRAINT "primary" TO "index137_163";
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=0c31bfd6 key=/Table/SystemConfigSpan/Start pri=0.03933799 epo=0 ts=1605267084.586286699,1 min=1605267084.003672247,0 seq=6} lock=true stat=PENDING rts=1605267084.003672247,0 wto=false max=1605267084.003672247,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE SCHEMA schema171 AUTHORIZATION root;
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   ALTER TABLE public.table137 ALTER COLUMN "col137_1" DROP DEFAULT;
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=45725ef5 key=/Table/SystemConfigSpan/Start pri=0.00530803 epo=0 ts=1605267084.586286699,1 min=1605267084.007182013,0 seq=6} lock=true stat=PENDING rts=1605267084.007182013,0 wto=false max=1605267084.007182013,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          |   DROP TABLE public.table137;
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=7f80e181 key=/Table/SystemConfigSpan/Start pri=0.00099321 epo=0 ts=1605267084.586286699,1 min=1605267084.120444793,0 seq=6} lock=true stat=PENDING rts=1605267084.120444793,0 wto=false max=1605267084.120444793,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE VIEW schema169.view170 AS SELECT col137_1, col137_0 FROM public.table161;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot create "schema169.view170" because the target database or schema does not exist (SQLSTATE 3F000)
          | BEGIN
          |   ALTER TABLE public.table137 ADD COLUMN col137_164 CHAR;
          | ***FAIL; Expected no errors, but got ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=0b33bf99 key=/Table/SystemConfigSpan/Start pri=0.00859252 epo=0 ts=1605267084.586286699,1 min=1605267084.253223995,0 seq=6} lock=true stat=PENDING rts=1605267084.253223995,0 wto=false max=1605267084.253223995,0 (SQLSTATE 40001)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2439944&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56379

(roachtest).schemachange/random-load failed on master@6b254f2d4a8e1470c1a7d5bd7c3f746d69de0797:

          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | OM public.table164;
          | ROLLBACK; expected SQLSTATE(S) 42P07, and got ERROR: relation "table164" already exists (SQLSTATE 42P07)
          | BEGIN
          |   DROP SCHEMA "schema124" CASCADE;
          |   DROP SEQUENCE "seq180";
          | ROLLBACK; ERROR: relation "seq180" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   DROP INDEX public.table110@"table110_col110_5_col110_2_col110_10_col110_11_col110_6_col110_8_col110_15_col110_3_col110_7_key";
          | ROLLBACK; ERROR: index "table110_col110_5_col110_2_col110_10_col110_11_col110_6_col110_8_col110_15_col110_3_col110_7_key" is in use as unique constraint (SQLSTATE 2BP01)
          | BEGIN
          |   ALTER TABLE public.table110 RENAME TO public.table166;
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=826b2424 key=/Table/SystemConfigSpan/Start pri=0.08565742 epo=0 ts=1605353549.854223689,1 min=1605353549.099765434,0 seq=6} lock=true stat=PENDING rts=1605353549.099765434,0 wto=false max=1605353549.099765434,0 (SQLSTATE 40001)
          | BEGIN
          |   DROP INDEX public.table110@"table110_col110_10_col110_11_key";
          | ROLLBACK; ERROR: index "table110_col110_10_col110_11_key" is in use as unique constraint (SQLSTATE 2BP01)
          | BEGIN
          |   ALTER TABLE public.table110 ALTER COLUMN "col110_15" SET DATA TYPE regclass;
          | ROLLBACK; ERROR: cannot convert BIT(24) to REGCLASS (SQLSTATE 42846)
          | BEGIN
          |   ALTER TABLE public.table110 ALTER COLUMN "col110_14" DROP STORED;
          | ROLLBACK; ERROR: column "col110_14" is not a computed column (SQLSTATE 42611)
          | BEGIN
          |   ALTER TABLE public.table110 DROP COLUMN "col110_13";
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=272d4320 key=/Table/SystemConfigSpan/Start pri=0.05886564 epo=0 ts=1605353549.854223689,1 min=1605353549.308744862,0 seq=7} lock=true stat=PENDING rts=1605353549.308744862,0 wto=false max=1605353549.308744862,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE UNIQUE INDEX index110_185 ON public.table110 (col110_13 DESC, col110_11 DESC, col110_2 DESC, col110_10 ASC, col110_5 DESC, col110_4 DESC) STORING (col110_12, col110_14, col110_9, col110_1);
          | ROLLBACK; ERROR: unimplemented: column col110_13 is of type oid[] and thus is not indexable (SQLSTATE 0A000)
          | BEGIN
          |   CREATE SCHEMA IF NOT EXISTS schema124 AUTHORIZATION root;
          | COMMIT;  
          | BEGIN
          |   SELECT 'validating all objects';
          |   ALTER TABLE public.table110 ALTER COLUMN "col110_13" DROP NOT NULL;
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=65f4a05a key=/Table/SystemConfigSpan/Start pri=0.00445382 epo=0 ts=1605353549.854223689,1 min=1605353549.328250054,0 seq=6} lock=true stat=PENDING rts=1605353549.328250054,0 wto=false max=1605353549.328250054,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER TABLE public.table110 RENAME COLUMN "col110_10" TO "col110_169";
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=71052954 key=/Table/SystemConfigSpan/Start pri=0.03664419 epo=0 ts=1605353549.854223689,1 min=1605353549.324675460,0 seq=6} lock=true stat=PENDING rts=1605353549.324675460,0 wto=false max=1605353549.324675460,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER TABLE public.table110 ADD COLUMN col110_167 NAME NOT NULL;
          | ***FAIL; Expected no errors, but got ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=a59290dc key=/Table/SystemConfigSpan/Start pri=0.01828331 epo=0 ts=1605353549.854223689,1 min=1605353549.168524846,0 seq=6} lock=true stat=PENDING rts=1605353549.168524846,0 wto=false max=1605353549.168524846,0 (SQLSTATE 40001)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2442839&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56379

(roachtest).schemachange/random-load failed on master@5d8f1d3dbc356573a1fb2d1d6bf564bf0370daff:

          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | ema581.table1285 AS SELECT col1252_11, col1252_3, col1252_9, col1252_4, col1252_8, col1252_6, col1252_2, col1252_1, col1252_10, col1252_11, col1252_5, col1252_8 FROM schema581.table1252, schema581.table1252;
          | ROLLBACK; expected SQLSTATE(S) 42712, and got ERROR: source name "table1252" specified more than once (missing AS clause) (SQLSTATE 42712)
          | BEGIN
          |   DROP SCHEMA "schema1191" CASCADE;
          | COMMIT;  
          | BEGIN
          |   DROP SEQUENCE "seq983";
          | COMMIT;  
          | BEGIN
          |   DROP VIEW public.view1288;
          | ROLLBACK; ERROR: relation "public.view1288" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   ALTER TABLE schema581.table1252 ALTER COLUMN "col1252_4" SET DATA TYPE timetz;
          | ROLLBACK; ERROR: ALTER COLUMN TYPE from varchar to timetz is only supported experimentally (SQLSTATE 0A000)
          |    23.0s        0           35.0           45.2     32.5    134.2    268.4    268.4 opOk
          |    23.0s        0            7.0            9.0    604.0    738.2    738.2    738.2 txnCmtErr
          |    23.0s        0            5.0            3.3    738.2   1409.3   1409.3   1409.3 txnOk
          |    23.0s        0           34.0           24.8      4.7     46.1    142.6    142.6 txnRbk
          | BEGIN
          |   DROP INDEX schema581.table1252@"table1252_col1252_4_col1252_7_col1252_9_col1252_6_col1252_8_col1252_2_idx";
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=c21a337a key=/Table/SystemConfigSpan/Start pri=0.02338681 epo=0 ts=1605441364.801467884,1 min=1605441364.415125650,0 seq=7} lock=true stat=PENDING rts=1605441364.415125650,0 wto=false max=1605441364.415125650,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER TABLE schema581.table1252 DROP CONSTRAINT "table1252_col1252_2_col1252_11_col1252_10_col1252_9_col1252_3_col1252_5_col1252_8_col1252_6_key";
          | ROLLBACK; ERROR: unimplemented: cannot drop UNIQUE constraint "table1252_col1252_2_col1252_11_col1252_10_col1252_9_col1252_3_col1252_5_col1252_8_col1252_6_key" using ALTER TABLE DROP CONSTRAINT, use DROP INDEX CASCADE instead (SQLSTATE 0A000)
          | BEGIN
          |   INSERT INTO schema581.table1252 ("col1252_0","col1252_1","col1252_2","col1252_3","col1252_4","col1252_5","col1252_6","col1252_7","col1252_8","col1252_9","col1252_10","col1252_11") VALUES ('177.73.145.210/21':::INET,'+Inf':::FLOAT8,'-54 years -780 days -11:41:04.285663':::INTERVAL,(-0.8617931604385376):::FLOAT8,e'T\x16;h`$\x03Y':::STRING,0.4509502029784501:::FLOAT8,'21:44:49.348625+01:21:00':::TIMETZ,1403836259:::OID,e'3\x12u':::STRING:::NAME,(-708040.0489512951885):::DECIMAL,'X':::STRING,NULL);
          |   DROP TABLE schema581.table1252;
          | ROLLBACK; ERROR: unimplemented: schema change statement cannot follow a statement that has written in the same transaction (SQLSTATE 0A000)
          | BEGIN
          |   ALTER TABLE schema581.table1252 ALTER COLUMN "col1252_1" SET NOT NULL;
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=a71c51c6 key=/Table/SystemConfigSpan/Start pri=0.00072541 epo=0 ts=1605441364.801467884,1 min=1605441364.523158868,0 seq=6} lock=true stat=PENDING rts=1605441364.523158868,0 wto=false max=1605441364.523158868,0 (SQLSTATE 40001)
          | BEGIN
          |   DROP INDEX schema581.table1252@"table1252_col1252_9_col1252_4_col1252_1_col1252_2_col1252_8_col1252_10_col1252_0_col1252_7_col1252_6_key";
          | ROLLBACK; ERROR: index "table1252_col1252_9_col1252_4_col1252_1_col1252_2_col1252_8_col1252_10_col1252_0_col1252_7_col1252_6_key" is in use as unique constraint (SQLSTATE 2BP01)
          | BEGIN
          |   ALTER TABLE schema581.table1252 ADD COLUMN col1252_1283 BYTES;
          | ***FAIL; Expected no errors, but got ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=2f36ed8d key=/Table/SystemConfigSpan/Start pri=0.03294689 epo=0 ts=1605441364.801467884,1 min=1605441364.419344634,0 seq=6} lock=true stat=PENDING rts=1605441364.419344634,0 wto=false max=1605441364.419344634,0 (SQLSTATE 40001)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2444337&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56379

(roachtest).schemachange/random-load failed on master@b4bf42c4fe10155717ac48f2e68ad8b501ca0b1a:

          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          |
          | BEGIN
          |   ALTER TABLE public.table265 ALTER COLUMN "col265_2" DROP STORED;
          | ROLLBACK; ERROR: column "col265_2" is not a computed column (SQLSTATE 42611)
          | BEGIN
          |   SELECT 'validating all objects';
          |   DROP SEQUENCE "seq304";
          |   ALTER TABLE public.table265 DROP CONSTRAINT "table265_col265_0_key";
          | ROLLBACK; ERROR: unimplemented: cannot drop UNIQUE constraint "table265_col265_0_key" using ALTER TABLE DROP CONSTRAINT, use DROP INDEX CASCADE instead (SQLSTATE 0A000)
          | BEGIN
          |   CREATE TABLE schema322.table336 AS SELECT public.table265.col265_2, public.table265.col265_1, public.table265.col265_0, public.table265.col265_1, public.table265.col265_1, public.table265.col265_2, public.table265.col265_0 FROM public.table265, public.table265, public.table265;
          | ROLLBACK; expected SQLSTATE(S) 42701,42712, and got ERROR: source name "table265" specified more than once (missing AS clause) (SQLSTATE 42712)
          | BEGIN
          |   CREATE TYPE "public.enum319" AS ENUM ('afa', 'fsaa');
          |   SELECT 'validating all objects';
          |   INSERT INTO public.table265 ("col265_0","col265_1","col265_2") VALUES (B'0000000000000000000000000000000000000000000000000000000000000000',37880245:::OID,'1970-01-05 08:45:02.000972':::TIMESTAMP),(B'101001010010011011001100001100001',2638068816:::OID,'0001-01-01 00:00:00':::TIMESTAMP),(B'001111011011100101010010000110001',3968208199:::OID,'1970-01-11 08:47:29.000393':::TIMESTAMP),(B'110100110001110101010001000111110',1940225272:::OID,'1970-01-03 15:48:51.000252':::TIMESTAMP),(B'011001100001110101100101110101010',1950879585:::OID,'1970-01-08 05:13:00.000906':::TIMESTAMP),(B'000000010111001110011111000011101',4068998745:::OID,'1970-01-09 12:35:58.000105':::TIMESTAMP),(B'111111100010110000001101010110011',1689631574:::OID,'1970-01-05 14:05:20.000614':::TIMESTAMP),(B'011001011000111110100100001110100',1400647460:::OID,'1970-01-10 22:10:34.000416':::TIMESTAMP),(B'001010101001100101101111100101011',1762098883:::OID,'1970-01-11 03:40:48.000385':::TIMESTAMP),(B'010001010001100110011010100100110',2848189602:::OID,'1970-01-11 04:39:30.000661':::TIMESTAMP);
          | ROLLBACK; ERROR: bit string length 64 does not match type BIT(33) (SQLSTATE 22026)
          | BEGIN
          |   ALTER TABLE public.table265 ALTER COLUMN "col265_2" DROP DEFAULT;
          |   CREATE SEQUENCE "seq335";
          |   ALTER TABLE public.table265 RENAME CONSTRAINT "table265_col265_0_col265_2_key" TO "index265_341";
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=962a3cf7 key=/Table/SystemConfigSpan/Start pri=0.00327751 epo=0 ts=1605526035.872477173,1 min=1605526035.354152817,0 seq=27} lock=true stat=PENDING rts=1605526035.354152817,0 wto=false max=1605526035.854152817,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER SEQUENCE "seq56" RENAME TO "seq329";
          |   CREATE TYPE "public.enum333" AS ENUM ('sd', 'sas', 'dsdd', 'aadas', 'dss');
          |   ALTER SEQUENCE "seq153" RENAME TO "seq337";
          |   ALTER TABLE public.table265 ALTER COLUMN "col265_1" SET DATA TYPE bit;
          | ROLLBACK; ERROR: cannot convert REGCLASS to BIT (SQLSTATE 42846)
          | BEGIN
          |   ALTER TABLE public.table265 ALTER COLUMN "col265_2" SET NOT NULL;
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE SEQUENCE "seq342";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   ALTER TABLE public.table265 ADD COLUMN col265_312 BYTES;
          |   ALTER TABLE public.table265 ADD COLUMN col265_343 REGNAMESPACE NULL;
          |   DROP SEQUENCE "seq153";
          |   CREATE SCHEMA IF NOT EXISTS schema344 AUTHORIZATION root;
          | ***FAIL; Expected no errors, but got ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=a655c7ca key=/Table/SystemConfigSpan/Start pri=0.01022411 epo=0 ts=1605526036.070005852,1 min=1605526035.386238532,0 seq=36} lock=true stat=PENDING rts=1605526035.386238532,0 wto=false max=1605526035.386238532,0 (SQLSTATE 40001)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2445673&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56379

(roachtest).schemachange/random-load failed on master@06944cb1996ed9ccfa036118b810e46673fe0ff4:

          | main.registerSchemaChangeRandomLoad.func1
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/schemachange_random_load.go:39
          | main.(*testRunner).runTest.func2
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:755
          | runtime.goexit
          |     /usr/local/go/src/runtime/asm_amd64.s:1374
        Wraps: (2) output in run_115309.990_n1_workload_run_schemachange
        Wraps: (3) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2453711-1605682855-70-n3cpu4:1 -- ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 returned
          | stderr:
          | I201118 11:53:10.734199 1 workload/cli/run.go:356  creating load generator...
          | I201118 11:53:10.843805 1 workload/cli/run.go:387  creating load generator... done (took 109.604656ms)
          | Error: ***UNEXPECTED ERROR: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=5abaf57c key=/Table/SystemConfigSpan/Start pri=0.02196170 epo=0 ts=1605700391.083482405,1 min=1605700391.016865422,0 seq=6} lock=true stat=PENDING rts=1605700391.016865422,0 wto=false max=1605700391.016865422,0 (SQLSTATE 40001)
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   ALTER TABLE public.table4 RENAME COLUMN "IrrelevantColumnName" TO "OtherIrrelevantName";
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table4" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   ALTER TABLE public.table5 ALTER COLUMN "IrrelevantColumnName" DROP NOT NULL;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table5" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE SCHEMA schema3 AUTHORIZATION root;
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=998ca344 key=/Table/SystemConfigSpan/Start pri=0.03158436 epo=0 ts=1605700390.853436124,1 min=1605700390.844496065,0 seq=13} lock=true stat=PENDING rts=1605700390.844496065,0 wto=false max=1605700391.344496065,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   ALTER TABLE public.table10 ALTER COLUMN "IrrelevantColumnName" DROP NOT NULL;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table10" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   ALTER SEQUENCE "seq14" RENAME TO "seq15";
          | ROLLBACK; ERROR: relation "seq14" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE SCHEMA IF NOT EXISTS schema12 AUTHORIZATION root;
          | ***FAIL; Expected no errors, but got ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=5abaf57c key=/Table/SystemConfigSpan/Start pri=0.02196170 epo=0 ts=1605700391.083482405,1 min=1605700391.016865422,0 seq=6} lock=true stat=PENDING rts=1605700391.016865422,0 wto=false max=1605700391.016865422,0 (SQLSTATE 40001)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2453711&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56379

(roachtest).schemachange/random-load failed on master@8ff64f5d897a9c7b39ca437193a11a7300cecbc1:

          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/schemachange_random_load.go:129
          | main.registerSchemaChangeRandomLoad.func1
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/schemachange_random_load.go:39
          | main.(*testRunner).runTest.func2
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:755
          | runtime.goexit
          |     /usr/local/go/src/runtime/asm_amd64.s:1374
        Wraps: (2) output in run_115946.224_n1_workload_run_schemachange
        Wraps: (3) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2457321-1605769144-69-n3cpu4:1 -- ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 returned
          | stderr:
          | I201119 11:59:46.962211 1 workload/cli/run.go:356  creating load generator...
          | I201119 11:59:47.044376 1 workload/cli/run.go:387  creating load generator... done (took 82.163828ms)
          | Error: ***UNEXPECTED ERROR: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=0e28df69 key=/Table/SystemConfigSpan/Start pri=0.01768023 epo=0 ts=1605787187.679084237,1 min=1605787187.463783189,0 seq=6} lock=true stat=PENDING rts=1605787187.463783189,0 wto=false max=1605787187.463783189,0 (SQLSTATE 40001)
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          |   CREATE SCHEMA IF NOT EXISTS schema8 AUTHORIZATION root;
          |   ALTER TABLE schema8.table34 ALTER COLUMN "IrrelevantColumnName" DROP NOT NULL;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "schema8.table34" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE SEQUENCE "seq18";
          |   CREATE SEQUENCE "seq30";
          |   CREATE TABLE public.table35 (col35_0 INT8 NOT NULL, col35_1 REGPROC NOT NULL, col35_2 INT2 NOT NULL, col35_3 UUID NOT NULL, col35_4 CHAR NOT NULL, col35_5 REGPROC NOT NULL, col35_6 "char"[] NOT NULL, col35_7 NAME NOT NULL, col35_8 TIMETZ NOT NULL, col35_9 "char" NOT NULL, col35_10 INT4, col35_11 BOX2D NOT NULL, PRIMARY KEY (col35_8, col35_5, col35_11, col35_4 ASC, col35_3, col35_2, col35_0, col35_7 DESC), UNIQUE (col35_8, col35_1 DESC, col35_11 DESC, col35_4), UNIQUE (col35_1, col35_9, col35_3, col35_10, col35_4 ASC, col35_11, col35_2 DESC), UNIQUE (col35_7, col35_8 ASC, col35_5 DESC), INDEX (col35_7 DESC, col35_1, col35_0 DESC, col35_4 ASC, col35_10 ASC, col35_8 DESC, col35_3 DESC, col35_5 ASC), UNIQUE (col35_10 ASC, col35_7 ASC, col35_8, col35_3 DESC, col35_1, col35_4 ASC, col35_2 DESC, col35_11 ASC, col35_5 ASC, col35_0 DESC, col35_9 ASC), FAMILY (col35_5, col35_8, col35_2), FAMILY (col35_6), FAMILY (col35_9, col35_11), FAMILY (col35_10), FAMILY (col35_7), FAMILY (col35_1, col35_3, col35_0), FAMILY (col35_4));
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=fad5a350 key=/Table/SystemConfigSpan/Start pri=0.03585713 epo=0 ts=1605787187.046967604,1 min=1605787187.045958412,0 seq=23} lock=true stat=PENDING rts=1605787187.045958412,0 wto=false max=1605787187.545958412,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE SEQUENCE "seq37";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE SCHEMA IF NOT EXISTS schema17 AUTHORIZATION root;
          |   SELECT 'validating all objects';
          |   CREATE TABLE IF NOT EXISTS schema17.table38 (col38_0 INET NOT NULL, col38_1 "char" NOT NULL, col38_2 DATE NOT NULL, col38_3 INTERVAL NOT NULL, col38_4 BOX2D NOT NULL, col38_5 VARCHAR NOT NULL, col38_6 TIMESTAMPTZ NOT NULL, col38_7 REGPROC NULL, col38_8 INET NOT NULL, col38_9 STRING NOT NULL, col38_10 DATE NOT NULL, col38_11 REGNAMESPACE NOT NULL, col38_12 INT2 NOT NULL, col38_13 REGNAMESPACE NOT NULL, col38_14 FLOAT8 NOT NULL, col38_15 UUID, col38_16 REGTYPE NOT NULL, PRIMARY KEY (col38_2, col38_16 DESC, col38_5, col38_14 ASC, col38_1, col38_0 DESC, col38_10, col38_3, col38_4, col38_6 ASC, col38_12 ASC, col38_9 ASC, col38_8 DESC, col38_11 ASC), FAMILY (col38_14, col38_11, col38_16, col38_2, col38_3), FAMILY (col38_6), FAMILY (col38_0), FAMILY (col38_4), FAMILY (col38_13), FAMILY (col38_5), FAMILY (col38_12), FAMILY (col38_15), FAMILY (col38_1, col38_10), FAMILY (col38_8), FAMILY (col38_7, col38_9));
          |   ALTER TABLE schema17.table38 ALTER COLUMN "col38_2" SET NOT NULL;
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=195513d3 key=/Table/SystemConfigSpan/Start pri=0.02515716 epo=0 ts=1605787187.463783189,1 min=1605787187.046578151,0 seq=31} lock=true stat=PENDING rts=1605787187.046578151,0 wto=false max=1605787187.546578151,0 (SQLSTATE 40001)
          | BEGIN
          |   DROP SCHEMA "schema43" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: unknown schema "schema43" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE SCHEMA IF NOT EXISTS schema42 AUTHORIZATION root;
          | ***FAIL; Expected no errors, but got ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=0e28df69 key=/Table/SystemConfigSpan/Start pri=0.01768023 epo=0 ts=1605787187.679084237,1 min=1605787187.463783189,0 seq=6} lock=true stat=PENDING rts=1605787187.463783189,0 wto=false max=1605787187.463783189,0 (SQLSTATE 40001)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2457321&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) tracked by #57487

(roachtest).schemachange/random-load failed on master@609cad52785f49338f7ceff289486b87db4cecb6:

          | main.runSchemaChangeRandomLoad
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/schemachange_random_load.go:129
          | main.registerSchemaChangeRandomLoad.func1
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/schemachange_random_load.go:39
          | main.(*testRunner).runTest.func2
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:755
          | runtime.goexit
          |     /usr/local/go/src/runtime/asm_amd64.s:1374
        Wraps: (2) output in run_111608.722_n1_workload_run_schemachange
        Wraps: (3) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2460636-1605855589-24-n3cpu4:1 -- ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 returned
          | stderr:
          | I201120 11:16:09.470930 1 workload/cli/run.go:357 ⋮ creating load generator...
          | I201120 11:16:09.563102 1 workload/cli/run.go:388 ⋮ creating load generator... done (took 92.170711ms)
          | Error: Class XX - Internal Error: ERROR: descriptor is being dropped (SQLSTATE XXUUU)
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | 9 days -12:09:20.775597':::INTERVAL,'1994-12-21 23:06:59.000859+00:00':::TIMESTAMPTZ,'-42 years -2 mons -644 days -23:59:35.960183':::INTERVAL,'BOX(-0.512757430069225 -0.5707332067596403,0.5557021461549285 1.510857191787701)':::BOX2D,'\x8e5d15c3c0':::BYTES,e'\x15n|J#':::STRING,(-566035382):::INT8,0.9254836440086365:::FLOAT8,'1972-02-17 19:01:57.000837':::TIMESTAMP,1251359798:::OID),('08:53:07.393016+01:48:00':::TIMETZ,'BOX(-10 -10,10 10)':::BOX2D,3627004707:::OID,(-11498):::INT8,(-5.950799359455295396E+34):::DECIMAL,(-515812449):::INT8,'52 years 8 mons 24 days 07:50:09.500249':::INTERVAL,'2003-06-08 05:02:33.00074+00:00':::TIMESTAMPTZ,'27 years 3 mons 166 days 07:13:44.217181':::INTERVAL,'BOX(-1.8859993067593757 -1.7100443851264524,-1.59481290899646 0.5300762648400374)':::BOX2D,'\x548699ac1a2384':::BYTES,'2':::STRING,(-1396291966):::INT8,'NaN':::FLOAT8,'1977-02-16 11:46:10.000468':::TIMESTAMP,2813931238:::OID),('11:25:09.927168+06:36:00':::TIMETZ,'BOX(-0.3402566344013556 0.5369485647445764,0.2784661835715944 0.6710991807514625)':::BOX2D,1656738929:::OID,(-24829):::INT8,(-1.781073272250936960E+26):::DECIMAL,231731393:::INT8,'-78 years -1 mons -822 days -16:26:28.4542':::INTERVAL,'2026-06-14 13:09:24.000349+00:00':::TIMESTAMPTZ,'-3 years -11 mons -833 days -13:48:46.218679':::INTERVAL,'BOX(-0.3087820639716118 -0.1936899362799954,0.8849590500794322 -0.16839992331042497)':::BOX2D,'\x0b90f2b560':::BYTES,e'i]0\\w$6d':::STRING,(-542542984):::INT8,NULL,'2012-01-07 05:37:46.000607':::TIMESTAMP,2192894445:::OID),('08:14:01.931489-05:17:00':::TIMETZ,'BOX(-0.1364937370039162 -1.3131825823658323,-0.09917275724009578 -0.1119184666226869)':::BOX2D,839344185:::OID,2972:::INT8,(-48993.26175498718083):::DECIMAL,260585206:::INT8,'54 years 8 mons 867 days 09:31:57.971866':::INTERVAL,'0001-01-01 00:00:00+00:00':::TIMESTAMPTZ,'290 years':::INTERVAL,'BOX(-1.2963010486951092 -0.5420389209387362,0.9452612262859554 0.27646931893348275)':::BOX2D,'\x47ecf3a2':::BYTES,e't|p\bCQMH/':::STRING,21588294:::INT8,(-1.1751652956008911):::FLOAT8,'1984-10-16 22:21:43.000954':::TIMESTAMP,3957229553:::OID);
          | ROLLBACK; ERROR: value too long for type CHAR (column "col258_11") (SQLSTATE 22001)
          | BEGIN
          |   ALTER TABLE public.table258 ALTER COLUMN "col258_12" DROP STORED;
          | ROLLBACK; ERROR: column "col258_12" is not a computed column (SQLSTATE 42611)
          | BEGIN
          |   ALTER SEQUENCE "seq174" RENAME TO "seq382";
          |   ALTER TABLE public.table258 ALTER COLUMN "col258_4" SET DATA TYPE bool;
          | ROLLBACK; ERROR: ALTER COLUMN TYPE from decimal to bool is only supported experimentally (SQLSTATE 0A000)
          | BEGIN
          |   ALTER TABLE public.table258 DROP CONSTRAINT "table258_col258_1_col258_3_col258_9_col258_7_col258_11_col258_5_col258_15_col258_0_col258_6_col258_12_col258_10_col258_2_key";
          | ROLLBACK; ERROR: unimplemented: cannot drop UNIQUE constraint "table258_col258_1_col258_3_col258_9_col258_7_col258_11_col258_5_col258_15_col258_0_col258_6_col258_12_col258_10_col258_2_key" using ALTER TABLE DROP CONSTRAINT, use DROP INDEX CASCADE instead (SQLSTATE 0A000)
          | BEGIN
          |   CREATE VIEW schema361.view392 AS SELECT public.table258.col258_5, public.table258.col258_1, public.table258.col258_0, public.table258.col258_4, public.table258.col258_11, public.table258.col258_10, public.table258.col258_13, public.table258.col258_6, public.table258.col258_2, public.table258.col258_8, public.table258.col258_9, public.table258.col258_12, public.table258.col258_11, public.table258.col258_4, public.table258.col258_10, public.table258.col258_14, public.table258.col258_1, public.table258.col258_5, public.table258.col258_13, public.table258.col258_9, public.table258.col258_3, public.table258.col258_7, public.table258.col258_2, public.table258.col258_8 FROM public.table258, public.table258;
          | ROLLBACK; expected SQLSTATE(S) 42701,42712, and got ERROR: source name "table258" specified more than once (missing AS clause) (SQLSTATE 42712)
          | BEGIN
          |   SELECT 'validating all objects';
          | COMMIT;  
          | BEGIN
          |   DROP TABLE public.table258;
          |   CREATE SEQUENCE "seq387";
          |   DROP TABLE schema280.table395;
          | ROLLBACK; ERROR: descriptor is being dropped (SQLSTATE XXUUU)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2460636&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) this is being tracked by https://github.com/cockroachdb/cockroach/issues/57476

(roachtest).schemachange/random-load failed on master@7b8288b26c3aad649ed6e2d89f679d46b5f3988d:

          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | ll objects';
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=03fc153a key=/Table/SystemConfigSpan/Start pri=0.03397418 epo=0 ts=1606218166.814985836,1 min=1606218166.809439197,0 seq=8} lock=true stat=PENDING rts=1606218166.809439197,0 wto=false max=1606218167.309439197,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          |   SELECT 'validating all objects';
          |   CREATE TYPE "public.enum7" AS ENUM ('saa', 'ad', 'asdf', 'sdf', 'ffasf');
          |   CREATE SEQUENCE "seq22";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE SCHEMA IF NOT EXISTS schema5 AUTHORIZATION root;
          |   SELECT 'validating all objects';
          |   CREATE SEQUENCE "seq24";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   CREATE TABLE public.table4 (col4_0 FLOAT4 NULL, col4_1 UUID, col4_2 INT2, col4_3 REGCLASS NOT NULL, col4_4 INT8 NOT NULL, col4_5 TIMETZ NOT NULL, col4_6 CHAR NULL, col4_7 REGNAMESPACE NOT NULL, PRIMARY KEY (col4_5 DESC, col4_7 ASC, col4_4), INDEX (col4_0 DESC, col4_2 DESC), INDEX (col4_3, col4_5 ASC, col4_4 ASC, col4_1 DESC) STORING (col4_0, col4_2, col4_6), UNIQUE (col4_0 DESC, col4_6, col4_3, col4_5 ASC, col4_1 ASC) STORING (col4_2), UNIQUE (col4_7, col4_4, col4_5, col4_6 ASC, col4_2, col4_1), UNIQUE (col4_4 ASC, col4_6 DESC, col4_5, col4_7 ASC, col4_3 DESC), INDEX (col4_7), INDEX (col4_0, col4_6 ASC, col4_3 DESC, col4_1 DESC, col4_4 ASC) STORING (col4_2), FAMILY (col4_7), FAMILY (col4_5, col4_6, col4_3, col4_0, col4_4, col4_1), FAMILY (col4_2));
          |   CREATE TABLE IF NOT EXISTS public.table8 (col8_0 TIMESTAMPTZ NOT NULL, col8_1 REGTYPE NOT NULL, col8_2 NAME NOT NULL, col8_3 REGPROC NOT NULL, col8_4 REGCLASS, col8_5 FLOAT4 NOT NULL, col8_6 FLOAT8 NOT NULL, col8_7 FLOAT8 NOT NULL, col8_8 BOOL NULL, col8_9 REGTYPE NOT NULL, col8_10 REGPROCEDURE NOT NULL, col8_11 FLOAT8 NOT NULL, col8_12 BYTES NOT NULL, col8_13 REGTYPE NOT NULL, col8_14 TIMETZ NOT NULL, col8_15 DECIMAL NOT NULL, PRIMARY KEY (col8_11 DESC, col8_3 DESC, col8_5 ASC, col8_15 ASC, col8_13, col8_7 DESC, col8_14 ASC, col8_6, col8_12 ASC, col8_0 ASC, col8_1 DESC, col8_10 ASC), INDEX (col8_5) STORING (col8_4, col8_8), INDEX (col8_6, col8_11, col8_5, col8_0 ASC, col8_3 ASC, col8_15, col8_7, col8_14 DESC, col8_12 ASC, col8_10 DESC, col8_9) STORING (col8_2, col8_4), INDEX (col8_8, col8_1, col8_4 DESC, col8_15 ASC, col8_5 DESC, col8_3, col8_13 ASC, col8_12, col8_6 ASC, col8_0 ASC, col8_11, col8_9 ASC) STORING (col8_2), INDEX (col8_4 ASC, col8_14 DESC, col8_7 DESC, col8_0 ASC, col8_5 ASC, col8_8, col8_12, col8_2 ASC), INDEX (col8_0 DESC, col8_14, col8_4 DESC, col8_10), FAMILY (col8_4), FAMILY (col8_0, col8_15, col8_14, col8_10, col8_2), FAMILY (col8_12, col8_11, col8_7), FAMILY (col8_5), FAMILY (col8_1, col8_9), FAMILY (col8_8), FAMILY (col8_3), FAMILY (col8_6, col8_13));
          |   INSERT INTO public.table8 ("col8_0","col8_1","col8_2","col8_3","col8_4","col8_5","col8_6","col8_7","col8_8","col8_9","col8_10","col8_11","col8_12","col8_13","col8_14","col8_15") VALUES ('1976-07-18 03:24:39.000892+00:00':::TIMESTAMPTZ,0:::OID,'9':::STRING:::NAME,3838553258:::OID,2989155138:::OID,0.3274061381816864:::FLOAT8,1.3722311109134768:::FLOAT8,0.32501392553827:::FLOAT8,true,1390999328:::OID,1745471069:::OID,1.1913592546406382:::FLOAT8,'\xc8':::BYTES,455306906:::OID,'05:32:06.87834+11:45:00':::TIMETZ,(-1.377937834229212485E+33):::DECIMAL),('1987-08-13 12:21:32.000255+00:00':::TIMESTAMPTZ,576838145:::OID,e'\x00':::STRING,3427710308:::OID,153244245:::OID,(-1.0776984691619873):::FLOAT8,1.9678954318582775:::FLOAT8,(-0.5330956597888308):::FLOAT8,true,182393763:::OID,1087501764:::OID,1.5761812743674164:::FLOAT8,'\x':::BYTES,4139404358:::OID,'21:17:16.5554+05:24:00':::TIMETZ,2.993806415238394832E+31:::DECIMAL);
          |   ALTER TABLE public.table4 ALTER COLUMN "col4_0" DROP DEFAULT;
          | Expected SQLSTATE(S) 0A000, but got no errors
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError

    cluster.go:1637,context.go:140,cluster.go:1626,test_runner.go:841: dead node detection: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod monitor teamcity-2468817-1606201101-24-n3cpu4 --oneshot --ignore-empty-nodes: exit status 1 3: 18992
        2: 18975
        1: dead
        Error: UNCLASSIFIED_PROBLEM: 1: dead
        (1) UNCLASSIFIED_PROBLEM
        Wraps: (2) attached stack trace
          -- stack trace:
          | main.glob..func14
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1147
          | main.wrap.func1
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:271
          | github.com/spf13/cobra.(*Command).execute
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:830
          | github.com/spf13/cobra.(*Command).ExecuteC
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:914
          | github.com/spf13/cobra.(*Command).Execute
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:864
          | main.main
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachprod/main.go:1850
          | runtime.main
          |     /usr/local/go/src/runtime/proc.go:204
          | runtime.goexit
          |     /usr/local/go/src/runtime/asm_amd64.s:1374
        Wraps: (3) 1: dead
        Error types: (1) errors.Unclassified (2) *withstack.withStack (3) *errutil.leafError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2468817&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56379

(roachtest).schemachange/random-load failed on master@d251e175421d9303492a4923fb933515987163b6:

          | Error: ***UNEXPECTED ERROR: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_PUSHER_ABORTED): "sql txn" meta={id=7a24f390 key=/Table/SystemConfigSpan/Start pri=0.01611002 epo=0 ts=1606305126.612429626,1 min=1606305126.406057244,0 seq=16} lock=true stat=ABORTED rts=1606305126.406057244,0 wto=false max=1606305126.406057244,0 (SQLSTATE 40001)
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | .00446904 epo=0 ts=1606305126.612429626,1 min=1606305126.428778274,0 seq=8} lock=true stat=PENDING rts=1606305126.428778274,0 wto=false max=1606305126.928778274,0 (SQLSTATE 40001)
          | BEGIN
          |   DROP SEQUENCE "seq90";
          |   SELECT 'validating all objects';
          |   SELECT 'validating all objects';
          |   ALTER VIEW public.view156 RENAME TO public.view157;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.view156" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   ALTER SEQUENCE "seq90" RENAME TO "seq137";
          |   CREATE SEQUENCE "seq159";
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=54bbd0c5 key=/Table/SystemConfigSpan/Start pri=0.01953412 epo=0 ts=1606305126.612429626,1 min=1606305126.274792240,0 seq=20} lock=true stat=PENDING rts=1606305126.274792240,0 wto=false max=1606305126.774792240,0 (SQLSTATE 40001)
          | BEGIN
          |   DROP SEQUENCE "seq90";
          |   ALTER TABLE public.table160 ALTER COLUMN "IrrelevantColumnName" DROP DEFAULT;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table160" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   ALTER SEQUENCE "seq90" RENAME TO "seq138";
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE SEQUENCE "seq144";
          |   CREATE TYPE "public.enum146" AS ENUM ('as', 'ddsff', 'd', 'afaddd');
          |   CREATE TABLE IF NOT EXISTS public.table153 (col153_0 INT4 NOT NULL, col153_1 "char", col153_2 CHAR, PRIMARY KEY (col153_0 ASC), INDEX (col153_2 DESC, col153_1 DESC), INDEX (col153_0 DESC), UNIQUE (col153_2 ASC), INDEX (col153_1, col153_2 DESC, col153_0), UNIQUE (col153_2 DESC, col153_1), UNIQUE (col153_0, col153_1 ASC, col153_2), UNIQUE (col153_2 ASC), UNIQUE (col153_2 DESC), INDEX (col153_1 DESC, col153_2 DESC, col153_0 ASC));
          |   ALTER TABLE public.table153 RENAME COLUMN "col153_0" TO "col153_155";
          |   ALTER SEQUENCE "seq90" RENAME TO "seq158";
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_PUSHER_ABORTED): "sql txn" meta={id=3d5d06f4 key=/Table/SystemConfigSpan/Start pri=0.00870307 epo=0 ts=1606305126.612429626,1 min=1606305126.479143155,0 seq=40} lock=true stat=ABORTED rts=1606305126.479143155,0 wto=false max=1606305126.479143155,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE TABLE public.table139 (col139_0 REGCLASS, col139_1 REGPROC NOT NULL, col139_2 DECIMAL NOT NULL, col139_3 FLOAT8 NOT NULL, col139_4 NAME NOT NULL, col139_5 TIME NOT NULL, col139_6 UUID, col139_7 TIMETZ NOT NULL, col139_8 NAME NOT NULL, col139_9 UUID NOT NULL, col139_10 BIT(23) NOT NULL, col139_11 INT8 NULL, PRIMARY KEY (col139_8 DESC, col139_2 DESC, col139_3 ASC, col139_10, col139_1 ASC, col139_7), INDEX (col139_9, col139_0, col139_6, col139_3 DESC, col139_7 ASC, col139_2 ASC, col139_10) STORING (col139_5, col139_11));
          |   CREATE UNIQUE INDEX IF NOT EXISTS index139_150 ON public.table139 (col139_10 DESC, col139_1 DESC, col139_7 DESC, col139_3 DESC, col139_4, col139_5, col139_2, col139_11 DESC, col139_0 DESC, col139_6 ASC);
          |   CREATE TYPE "schema151.enum152" AS ENUM ('a', 'adds', 'ad', 'ffs');
          |   DROP SEQUENCE "seq90";
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_PUSHER_ABORTED): "sql txn" meta={id=209efde8 key=/Table/SystemConfigSpan/Start pri=0.02723965 epo=0 ts=1606305126.612429626,1 min=1606305126.472084643,0 seq=32} lock=true stat=ABORTED rts=1606305126.472084643,0 wto=false max=1606305126.472084643,0 (SQLSTATE 40001)
          | BEGIN
          |   DROP SEQUENCE "seq90";
          |   CREATE SCHEMA IF NOT EXISTS schema161 AUTHORIZATION root;
          | ***FAIL; Expected no errors, but got ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_PUSHER_ABORTED): "sql txn" meta={id=7a24f390 key=/Table/SystemConfigSpan/Start pri=0.01611002 epo=0 ts=1606305126.612429626,1 min=1606305126.406057244,0 seq=16} lock=true stat=ABORTED rts=1606305126.406057244,0 wto=false max=1606305126.406057244,0 (SQLSTATE 40001)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2471879&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56379

(roachtest).schemachange/random-load failed on master@fe5da5c2df069dc7f820f5c3e2f3e03f1cb7b661:

          | runtime.goexit
          |     /usr/local/go/src/runtime/asm_amd64.s:1374
        Wraps: (2) output in run_114050.535_n1_workload_run_schemachange
        Wraps: (3) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2475210-1606374159-24-n3cpu4:1 -- ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 returned
          | stderr:
          | I201126 11:40:51.266511 1 workload/cli/run.go:357 ⋮ creating load generator...
          | I201126 11:40:51.346180 1 workload/cli/run.go:388 ⋮ creating load generator... done (took 79.66866ms)
          | Error: ***UNEXPECTED ERROR: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_PUSHER_ABORTED): "sql txn" meta={id=b2cdf495 key=/Table/SystemConfigSpan/Start pri=0.01249458 epo=0 ts=1606390851.347808054,0 min=1606390851.347808054,0 seq=14} lock=true stat=ABORTED rts=1606390851.347808054,0 wto=false max=1606390851.347808054,0 (SQLSTATE 40001)
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | l16_10 DESC, col16_8 DESC, col16_7, col16_9), INDEX (col16_15, col16_16, col16_3 DESC, col16_6 ASC, col16_1 ASC, col16_2 DESC, col16_9 DESC, col16_14, col16_4 DESC, col16_11, col16_5 ASC, col16_13) STORING (col16_0, col16_8));
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot create "schema15.table16" because the target database or schema does not exist (SQLSTATE 3F000)
          | BEGIN
          |   CREATE TABLE IF NOT EXISTS public.table10 (col10_0 "char", col10_1 REGPROCEDURE NULL, col10_2 REGCLASS NOT NULL, col10_3 REGPROCEDURE NOT NULL, col10_4 REGTYPE NOT NULL, col10_5 TIME NOT NULL, col10_6 BOOL NOT NULL, col10_7 "char" NOT NULL, col10_8 UUID NOT NULL, PRIMARY KEY (col10_4 ASC, col10_6, col10_7 DESC, col10_2, col10_5 ASC, col10_8 DESC, col10_3 ASC), INDEX (col10_7 ASC, col10_4, col10_5 ASC, col10_3), INDEX (col10_5 ASC, col10_0 DESC, col10_4, col10_1, col10_8, col10_3 ASC, col10_2 DESC), UNIQUE (col10_3 DESC, col10_0, col10_4 ASC, col10_1, col10_6 DESC, col10_7), FAMILY (col10_5, col10_6, col10_7, col10_1, col10_2, col10_0, col10_3, col10_4, col10_8));
          |   CREATE UNIQUE INDEX IF NOT EXISTS index10_19 ON public.table10 (col10_7, col10_1 DESC, col10_8, col10_0 DESC, col10_2, col10_5 DESC, col10_3) STORING (col10_4, col10_6);
          | ROLLBACK; ERROR: index "index10_19" already contains column "col10_4" (SQLSTATE 42701)
          | BEGIN
          |   CREATE TABLE IF NOT EXISTS public.table11 (col11_0 TIMESTAMPTZ NULL, col11_1 DATE[] NOT NULL, col11_2 OID NULL, col11_3 NAME NULL, col11_4 TIMESTAMPTZ NOT NULL, col11_5 OID, col11_6 BOOL[] NULL, col11_7 "char" NOT NULL, col11_8 DECIMAL NOT NULL, col11_9 FLOAT4 NOT NULL, col11_10 DECIMAL NULL, col11_11 REGPROC NULL, col11_12 INT4 NOT NULL, UNIQUE (col11_9 ASC, col11_8 ASC, col11_10 DESC, col11_3 DESC), INDEX (col11_2, col11_5 ASC) STORING (col11_0, col11_1, col11_3, col11_4, col11_6, col11_9, col11_11, col11_12), INDEX (col11_12, col11_4 DESC, col11_7), UNIQUE (col11_9 ASC, col11_0 DESC, col11_3, col11_12 ASC, col11_5, col11_8 ASC, col11_11 ASC, col11_10, col11_7 ASC, col11_2 ASC) STORING (col11_1), UNIQUE (col11_9, col11_12, col11_11 DESC, col11_2, col11_0, col11_4 ASC, col11_8, col11_5 ASC, col11_10) STORING (col11_1, col11_3, col11_7), UNIQUE (col11_11 ASC, col11_5 ASC, col11_0 ASC, col11_9 ASC, col11_7 ASC, col11_8 ASC), INDEX (col11_12 DESC, col11_4, col11_2 DESC, col11_5 ASC) STORING (col11_0, col11_7, col11_8, col11_9, col11_10));
          |   ALTER TABLE public.table18 ALTER COLUMN "IrrelevantColumnName" DROP NOT NULL;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table18" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE TYPE "public.enum6" AS ENUM ('fas', 'aadfsf', 'fsaaff', 'df');
          |   CREATE SEQUENCE "seq14";
          |   CREATE SEQUENCE "seq22";
          |   DROP SEQUENCE "seq14";
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=8d5fd098 key=/Table/SystemConfigSpan/Start pri=0.02311217 epo=0 ts=1606390851.347808054,1 min=1606390851.347206099,0 seq=38} lock=true stat=PENDING rts=1606390851.347206099,0 wto=false max=1606390851.847206099,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          |   SELECT 'validating all objects';
          |   ALTER TABLE public.table23 RENAME TO public.table24;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table23" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   SELECT 'validating all objects';
          |   DROP TABLE schema25.table26;
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   CREATE TYPE "public.enum3" AS ENUM ('fa', 'dd');
          |   CREATE SCHEMA IF NOT EXISTS schema9 AUTHORIZATION root;
          | ***FAIL; Expected no errors, but got ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_PUSHER_ABORTED): "sql txn" meta={id=b2cdf495 key=/Table/SystemConfigSpan/Start pri=0.01249458 epo=0 ts=1606390851.347808054,0 min=1606390851.347808054,0 seq=14} lock=true stat=ABORTED rts=1606390851.347808054,0 wto=false max=1606390851.347808054,0 (SQLSTATE 40001)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2475210&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56379

(roachtest).schemachange/random-load failed on master@361215163c597bd1460bba65ca3298f37e29aacc:

          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | 000, and got ERROR: unknown schema "schema18" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE SEQUENCE "seq24";
          |   CREATE TABLE IF NOT EXISTS public.table25 (col25_0 NAME NOT NULL, col25_1 STRING NULL, col25_2 REGCLASS NOT NULL, col25_3 REGNAMESPACE, col25_4 DECIMAL[] NULL, col25_5 REGNAMESPACE NOT NULL, col25_6 DECIMAL NOT NULL, col25_7 FLOAT4 NOT NULL, col25_8 INT4[] NULL, col25_9 NAME NOT NULL, PRIMARY KEY (col25_9 ASC, col25_6 DESC, col25_0 ASC, col25_7 ASC), UNIQUE (col25_9, col25_1 DESC, col25_7 ASC, col25_6, col25_0 ASC, col25_2, col25_3 DESC, col25_5), UNIQUE (col25_0 DESC, col25_1 DESC, col25_6 DESC, col25_7, col25_2, col25_9) STORING (col25_3, col25_4), INDEX (col25_5 ASC, col25_6 DESC, col25_7, col25_0 ASC), INDEX (col25_1 ASC, col25_0, col25_3, col25_6 DESC), UNIQUE (col25_5 ASC));
          |   ALTER TABLE public.table25 DROP COLUMN "col25_0";
          | ROLLBACK; expected SQLSTATE(S) 42P10, and got ERROR: column "col25_0" is referenced by the primary key (SQLSTATE 42P10)
          | BEGIN
          |   CREATE SCHEMA public AUTHORIZATION root;
          | ROLLBACK; expected SQLSTATE(S) 42P06, and got ERROR: schema "public" already exists (SQLSTATE 42P06)
          | BEGIN
          |   CREATE SEQUENCE "seq8";
          |   CREATE SCHEMA schema16 AUTHORIZATION root;
          |   CREATE SEQUENCE "seq34";
          |   DROP SCHEMA "schema16" CASCADE;
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=27d15b42 key=/Table/SystemConfigSpan/Start pri=0.03419584 epo=0 ts=1606478270.492409214,1 min=1606478270.104153276,0 seq=46} lock=true stat=PENDING rts=1606478270.104153276,0 wto=false max=1606478270.604153276,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE SEQUENCE "seq35";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE SCHEMA IF NOT EXISTS schema19 AUTHORIZATION root;
          |   CREATE SEQUENCE "seq39";
          |   SELECT 'validating all objects';
          |   ALTER SEQUENCE "seq40" RENAME TO "seq41";
          | ROLLBACK; ERROR: relation "seq40" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE SCHEMA IF NOT EXISTS schema22 AUTHORIZATION root;
          |   CREATE TABLE schema22.table42 (col42_0 INTERVAL NOT NULL, col42_1 BIT(30) NOT NULL, col42_2 INET NOT NULL, col42_3 DECIMAL NULL, col42_4 BIT(23) NOT NULL, col42_5 DATE NULL, col42_6 BOX2D NOT NULL, col42_7 BOX2D, col42_8 BOX2D, col42_9 UUID NOT NULL, PRIMARY KEY (col42_2 ASC, col42_6 DESC, col42_9 DESC, col42_1 DESC, col42_4 ASC, col42_0 DESC), UNIQUE (col42_4 ASC, col42_9 ASC, col42_6 ASC, col42_8, col42_5, col42_1, col42_2 DESC, col42_3 DESC), INDEX (col42_9 DESC, col42_0 ASC, col42_6 ASC, col42_7 ASC, col42_5 DESC) STORING (col42_8), FAMILY (col42_5), FAMILY (col42_6, col42_7, col42_1), FAMILY (col42_4, col42_8), FAMILY (col42_3), FAMILY (col42_0), FAMILY (col42_9, col42_2));
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=5ba2adec key=/Table/SystemConfigSpan/Start pri=0.06755334 epo=0 ts=1606478270.593662758,1 min=1606478270.104727374,0 seq=20} lock=true stat=PENDING rts=1606478270.104727374,0 wto=false max=1606478270.604727374,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE TYPE "public.enum3" AS ENUM ('da', 'aafas', 'ffsfd', 'afadd', 'd', 'ddaadf');
          |   CREATE SEQUENCE "seq43";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   CREATE SCHEMA schema37 AUTHORIZATION root;
          | ***FAIL; Expected no errors, but got ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=a2b763d2 key=/Table/SystemConfigSpan/Start pri=0.00117747 epo=0 ts=1606478270.696344811,1 min=1606478270.593662758,0 seq=6} lock=true stat=PENDING rts=1606478270.593662758,0 wto=false max=1606478270.593662758,0 (SQLSTATE 40001)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2477206&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56379

(roachtest).schemachange/random-load failed on master@54cdc5fec0e7dc835af7d2fc4231b52d49a71bf8:

          |     /usr/local/go/src/runtime/asm_amd64.s:1374
        Wraps: (2) output in run_120423.403_n1_workload_run_schemachange
        Wraps: (3) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2479006-1606547424-24-n3cpu4:1 -- ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 returned
          | stderr:
          | I201128 12:04:24.198110 1 workload/cli/run.go:357 ⋮ creating load generator...
          | I201128 12:04:24.280780 1 workload/cli/run.go:388 ⋮ creating load generator... done (took 82.667018ms)
          | Error: ***UNEXPECTED ERROR: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=ec845caa key=/Table/SystemConfigSpan/Start pri=0.01202333 epo=0 ts=1606565064.471477150,1 min=1606565064.282723310,0 seq=6} lock=true stat=PENDING rts=1606565064.282723310,0 wto=false max=1606565064.282723310,0 (SQLSTATE 40001)
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | BEGIN
          |   CREATE SEQUENCE "seq9";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   SELECT 'validating all objects';
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE SEQUENCE "seq4";
          |   DROP TABLE public.table10;
          | ROLLBACK; ERROR: relation "public.table10" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   SELECT 'validating all objects';
          | COMMIT;  
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE SEQUENCE "seq18";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   SELECT 'validating all objects';
          |   ALTER VIEW public.view14 RENAME TO public.view15;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.view14" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   SELECT 'validating all objects';
          | COMMIT;  
          | BEGIN
          |   CREATE SCHEMA IF NOT EXISTS schema11 AUTHORIZATION root;
          | ***FAIL; Expected no errors, but got ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=ec845caa key=/Table/SystemConfigSpan/Start pri=0.01202333 epo=0 ts=1606565064.471477150,1 min=1606565064.282723310,0 seq=6} lock=true stat=PENDING rts=1606565064.282723310,0 wto=false max=1606565064.282723310,0 (SQLSTATE 40001)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2479006&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56379

(roachtest).schemachange/random-load failed on master@76935fd62c5a76f88b754cd3f9a5bfb3ccf1d8c2:

          | runtime.goexit
          |     /usr/local/go/src/runtime/asm_amd64.s:1374
        Wraps: (2) output in run_120345.423_n1_workload_run_schemachange
        Wraps: (3) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2480413-1606719755-74-n3cpu4:1 -- ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 returned
          | stderr:
          | I201130 12:03:46.150418 1 workload/cli/run.go:357 ⋮ creating load generator...
          | I201130 12:03:46.220625 1 workload/cli/run.go:388 ⋮ creating load generator... done (took 70.205994ms)
          | Error: ***UNEXPECTED ERROR: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_PUSHER_ABORTED): "sql txn" meta={id=6b90d28c key=/Table/SystemConfigSpan/Start pri=0.00314829 epo=0 ts=1606737826.392779563,1 min=1606737826.230669667,0 seq=25} lock=true stat=ABORTED rts=1606737826.230669667,0 wto=false max=1606737826.230669667,0 (SQLSTATE 40001)
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | col37_17 DESC, col37_9 DESC) STORING (col37_0, col37_11), UNIQUE (col37_11 ASC, col37_10 ASC, col37_12), INDEX (col37_12 ASC, col37_15 ASC, col37_3 ASC, col37_8) STORING (col37_0, col37_2, col37_7, col37_9), UNIQUE (col37_14 DESC, col37_9, col37_10, col37_0 DESC, col37_17 ASC, col37_1 DESC, col37_12, col37_11, col37_3 DESC, col37_5 ASC), UNIQUE (col37_4 DESC, col37_13 ASC, col37_5 DESC, col37_8 ASC, col37_1 ASC, col37_10, col37_6 DESC) STORING (col37_0, col37_2, col37_9, col37_11, col37_12), FAMILY (col37_13), FAMILY (col37_1, col37_8, col37_10, col37_0), FAMILY (col37_9, col37_7), FAMILY (col37_14, col37_2), FAMILY (col37_4), FAMILY (col37_16), FAMILY (col37_15, col37_11, col37_6, col37_18, col37_5), FAMILY (col37_17), FAMILY (col37_3), FAMILY (col37_12));
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot create "schema36.table37" because the target database or schema does not exist (SQLSTATE 3F000)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE TABLE IF NOT EXISTS public.table30 (col30_0 REGCLASS, col30_1 OID NOT NULL, col30_2 DATE, INDEX (col30_0), UNIQUE (col30_0) STORING (col30_2), INDEX (col30_0 ASC, col30_1), UNIQUE (col30_0, col30_1, col30_2), UNIQUE (col30_1 ASC), INDEX (col30_0), UNIQUE (col30_1 DESC, col30_2, col30_0), FAMILY (col30_1), FAMILY (col30_0), FAMILY (col30_2));
          |   ALTER TABLE public.table30 DROP CONSTRAINT "table30_col30_0_col30_1_col30_2_key";
          | ROLLBACK; ERROR: unimplemented: cannot drop UNIQUE constraint "table30_col30_0_col30_1_col30_2_key" using ALTER TABLE DROP CONSTRAINT, use DROP INDEX CASCADE instead (SQLSTATE 0A000)
          | BEGIN
          |   SELECT 'validating all objects';
          |   SELECT 'validating all objects';
          |   CREATE SEQUENCE "seq38";
          |   DROP SCHEMA "schema39" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: unknown schema "schema39" (SQLSTATE 3F000)
          | BEGIN
          |   ALTER SEQUENCE "seq21" RENAME TO "seq27";
          |   CREATE TABLE IF NOT EXISTS public.table33 (col33_0 BOX2D NOT NULL, col33_1 OID NOT NULL, col33_2 DATE NULL, col33_3 FLOAT4, col33_4 REGPROCEDURE, col33_5 INT8 NOT NULL, col33_6 INT8 NOT NULL, PRIMARY KEY (col33_1 ASC), INDEX (col33_0 DESC, col33_1 DESC, col33_2 ASC, col33_5 ASC, col33_6 DESC), INDEX (col33_5, col33_1 ASC, col33_0) STORING (col33_2), UNIQUE (col33_3 ASC, col33_4 DESC) STORING (col33_0), UNIQUE (col33_3 ASC, col33_2 ASC, col33_5 DESC), INDEX (col33_2 DESC, col33_6 DESC), UNIQUE (col33_1, col33_2, col33_3 DESC), UNIQUE (col33_1 ASC, col33_6 DESC, col33_5, col33_2 DESC), UNIQUE (col33_4 DESC, col33_0 ASC, col33_5) STORING (col33_6));
          |   ALTER TABLE public.table33 ALTER COLUMN "col33_1" DROP DEFAULT;
          |   DROP TABLE public.table33;
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=ef2c6f74 key=/Table/SystemConfigSpan/Start pri=0.01184209 epo=0 ts=1606737826.392779563,1 min=1606737826.231153736,0 seq=40} lock=true stat=PENDING rts=1606737826.231153736,0 wto=false max=1606737826.731153736,0 (SQLSTATE 40001)
          | BEGIN
          |   DROP SEQUENCE "seq21";
          |   SELECT 'validating all objects';
          |   CREATE SEQUENCE "seq41";
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=1e972175 key=/Table/SystemConfigSpan/Start pri=0.01492044 epo=0 ts=1606737826.392779563,1 min=1606737826.233393908,0 seq=19} lock=true stat=PENDING rts=1606737826.233393908,0 wto=false max=1606737826.733393908,0 (SQLSTATE 40001)
          | BEGIN
          |   DROP SEQUENCE "seq21";
          |   CREATE TYPE "schema42.enum43" AS ENUM ('fda', 'ss');
          |   CREATE SCHEMA IF NOT EXISTS schema44 AUTHORIZATION root;
          | ***FAIL; Expected no errors, but got ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_PUSHER_ABORTED): "sql txn" meta={id=6b90d28c key=/Table/SystemConfigSpan/Start pri=0.00314829 epo=0 ts=1606737826.392779563,1 min=1606737826.230669667,0 seq=25} lock=true stat=ABORTED rts=1606737826.230669667,0 wto=false max=1606737826.230669667,0 (SQLSTATE 40001)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2480413&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56379

(roachtest).schemachange/random-load failed on master@22906f72e795f9f2c69828e65194f4177833ffbb:

          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | tart pri=0.00175764 epo=0 ts=1606824893.325463944,1 min=1606824893.115791577,0 seq=36} lock=true stat=PENDING rts=1606824893.115791577,0 wto=false max=1606824893.615791577,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          | COMMIT;  
          | BEGIN
          |   CREATE SEQUENCE "seq92";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   ALTER SEQUENCE "seq75" RENAME TO "seq88";
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   DROP SCHEMA "schema87" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: unknown schema "schema87" (SQLSTATE 3F000)
          | BEGIN
          |   SELECT 'validating all objects';
          |   ALTER TABLE public.table94 ALTER COLUMN "IrrelevantColumnName" DROP NOT NULL;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table94" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE SEQUENCE "seq99";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE TABLE public.table86 (col86_0 BOX2D NOT NULL, col86_1 FLOAT4 NOT NULL, col86_2 "char" NOT NULL, col86_3 BYTES NOT NULL, col86_4 BOX2D NOT NULL, col86_5 TIMETZ NOT NULL, col86_6 UUID, col86_7 BIT(46) NOT NULL, col86_8 BYTES NOT NULL, col86_9 BIT(15) NOT NULL, col86_10 BOX2D NOT NULL, PRIMARY KEY (col86_7 DESC, col86_4 ASC, col86_5 DESC, col86_0 ASC, col86_1, col86_9 DESC, col86_8, col86_3 DESC, col86_10), UNIQUE (col86_1 DESC, col86_10, col86_7 ASC, col86_3 DESC, col86_4 ASC, col86_8 ASC, col86_2, col86_5 DESC, col86_0) STORING (col86_6), INDEX (col86_9 ASC, col86_1 DESC, col86_5 ASC, col86_10 DESC, col86_0 DESC, col86_3 DESC, col86_8 ASC, col86_4, col86_6 DESC, col86_2 DESC, col86_7 ASC), INDEX (col86_6 ASC, col86_3 ASC, col86_7 ASC, col86_8, col86_4 ASC, col86_0 DESC, col86_5), INDEX (col86_2 DESC, col86_1, col86_0, col86_7 ASC), FAMILY (col86_7, col86_5, col86_9), FAMILY (col86_0), FAMILY (col86_8), FAMILY (col86_6, col86_3), FAMILY (col86_1, col86_10), FAMILY (col86_2, col86_4));
          |   CREATE VIEW public.view100 AS SELECT public.table86.col86_0, public.table86.col86_4, public.table86.col86_7 FROM public.table86, public.table86;
          | ROLLBACK; expected SQLSTATE(S) 42712, and got ERROR: source name "table86" specified more than once (missing AS clause) (SQLSTATE 42712)
          | BEGIN
          |   CREATE TYPE "schema90.enum91" AS ENUM ('fasa', 'ddafd');
          |   CREATE TABLE IF NOT EXISTS public.table97 (col97_0 BIT(45) NOT NULL, col97_1 BOX2D NOT NULL, col97_2 REGTYPE, col97_3 "char" NOT NULL, col97_4 VARCHAR NOT NULL, col97_5 BIT(26) NULL, col97_6 TIMETZ NULL, col97_7 TIMESTAMPTZ, col97_8 INT2 NOT NULL, PRIMARY KEY (col97_1 ASC, col97_0, col97_3 ASC, col97_4 ASC, col97_8 DESC), UNIQUE (col97_0 DESC, col97_7 DESC) STORING (col97_6), UNIQUE (col97_4, col97_8 ASC, col97_1, col97_0 ASC, col97_3), FAMILY (col97_5, col97_3, col97_1), FAMILY (col97_2, col97_0, col97_6, col97_7, col97_4), FAMILY (col97_8));
          |   ALTER TABLE public.table97 DROP COLUMN "col97_3";
          | ROLLBACK; expected SQLSTATE(S) 42P10, and got ERROR: column "col97_3" is referenced by the primary key (SQLSTATE 42P10)
          | BEGIN
          |   CREATE SCHEMA IF NOT EXISTS schema70 AUTHORIZATION root;
          |   CREATE SEQUENCE "seq103";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   CREATE SCHEMA IF NOT EXISTS schema77 AUTHORIZATION root;
          |   SELECT 'validating all objects';
          |   CREATE TYPE "public.enum109" AS ENUM ('fdf', 'fdfas');
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=bfe34e7d key=/Table/SystemConfigSpan/Start pri=0.05759913 epo=0 ts=1606824894.127679078,1 min=1606824893.045298927,0 seq=22} lock=true stat=PENDING rts=1606824893.045298927,0 wto=false max=1606824893.545298927,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE SCHEMA IF NOT EXISTS schema79 AUTHORIZATION root;
          | Expected SQLSTATE(S) 42P01, but got no errors
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2483915&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56379

(roachtest).schemachange/random-load failed on master@5af065eeb4f520ee93901e75fbef5d877f06585c:

          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | , 'afa', 'aafad', 'dfaa');
          |   DROP INDEX public.table52@"index52_142";
          | ROLLBACK; ERROR: index "index52_142" does not exist (SQLSTATE 42704)
          | BEGIN
          |   CREATE TABLE public.table144 AS SELECT public.table52.col52_1, public.table102.col102_9, public.table102.col102_10, public.table102.col102_3, public.table102.col102_13, public.table102.col102_17, public.table102.col102_6, public.table102.col102_14, public.table102.col102_7, public.table102.col102_11, public.table102.col102_8, public.table102.col102_5, public.table102.col102_1, public.table52.col52_1 FROM public.table52, public.table102, public.table52;
          | ROLLBACK; expected SQLSTATE(S) 42701,42712, and got ERROR: source name "table52" specified more than once (missing AS clause) (SQLSTATE 42712)
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   ALTER TABLE public.table102 ALTER COLUMN "col102_13" SET NOT NULL;
          |   CREATE SEQUENCE "seq138";
          |   ALTER TABLE public.table52 DROP CONSTRAINT "primary";
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_PUSHER_ABORTED): "sql txn" meta={id=ef8e9f19 key=/Table/SystemConfigSpan/Start pri=0.00166711 epo=0 ts=1606909307.644920709,1 min=1606909307.503951983,0 seq=24} lock=true stat=ABORTED rts=1606909307.503951983,0 wto=false max=1606909307.503951983,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          |   ALTER TABLE public.table102 RENAME TO public.table132;
          |   ALTER TABLE public.table132 ALTER COLUMN "col102_2" DROP DEFAULT;
          |   ALTER TABLE public.table52 ALTER COLUMN "col52_1" DROP NOT NULL;
          | ROLLBACK; expected SQLSTATE(S) 42P16, and got ERROR: column "col52_1" is in a primary index (SQLSTATE 42P16)
          | BEGIN
          |   DROP TABLE public.table52;
          |   ALTER TABLE public.table102 RENAME TO public.table146;
          |   ALTER TABLE public.table147 RENAME COLUMN "IrrelevantColumnName" TO "OtherIrrelevantName";
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table147" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   SELECT 'validating all objects';
          | COMMIT;  
          | BEGIN
          |   CREATE UNIQUE INDEX index102_150 ON public.table102 (col102_5 ASC, col102_13 ASC, col102_4 ASC, col102_1, col102_7 DESC, col102_6 DESC, col102_10 DESC, col102_8 DESC, col102_12 ASC, col102_2 ASC, col102_0, col102_16 ASC, col102_17 ASC, col102_9 ASC, col102_3, col102_14, col102_11 DESC, col102_18) STORING (col102_15);
          | ROLLBACK; ERROR: unimplemented: column col102_1 is of type float[] and thus is not indexable (SQLSTATE 0A000)
          | BEGIN
          |   ALTER VIEW schema139.view140 RENAME TO schema139.view141;
          | ROLLBACK; expected SQLSTATE(S) 3F000,42P01, and got ERROR: relation "schema139.view140" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   ALTER TABLE public.table102 ALTER COLUMN "col102_6" DROP NOT NULL;
          | ROLLBACK; expected SQLSTATE(S) 42P16, and got ERROR: column "col102_6" is in a primary index (SQLSTATE 42P16)
          | BEGIN
          |   DROP TABLE public.table102;
          | COMMIT;  ***FAIL: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=fac22ca1 key=/Table/SystemConfigSpan/Start pri=0.01288881 epo=0 ts=1606909307.909989342,1 min=1606909307.715310569,0 seq=11} lock=true stat=PENDING rts=1606909307.715310569,0 wto=false max=1606909308.215310569,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE VIEW public.view143 AS SELECT public.table102.col102_13, public.table102.col102_8, public.table52.col52_0, public.table52.col52_1 FROM public.table102, public.table52;
          | ***FAIL; Expected no errors, but got ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_PUSHER_ABORTED): "sql txn" meta={id=8cdcaf7b key=/Table/SystemConfigSpan/Start pri=0.00626780 epo=0 ts=1606909307.644920709,1 min=1606909307.446182097,0 seq=13} lock=true stat=ABORTED rts=1606909307.446182097,0 wto=false max=1606909307.446182097,0 (SQLSTATE 40001)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2488192&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) resolved by https://github.com/cockroachdb/cockroach/pull/56379

(roachtest).schemachange/random-load failed on master@677f6f89f97d492f5eb443fcdb326d5695ddc5d7:

          |
          | stdout:
          | BEGIN
          |   DROP VIEW public.view7;
          | ROLLBACK; ERROR: relation "public.view7" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   SELECT 'validating all objects';
          | COMMIT;  
          | BEGIN
          |   CREATE SEQUENCE "seq3";
          | COMMIT;  
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   ALTER TABLE schema1.table2 RENAME COLUMN "IrrelevantColumnName" TO "OtherIrrelevantName";
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "schema1.table2" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   SELECT 'validating all objects';
          | COMMIT;  
          | BEGIN
          |   ALTER TABLE schema17.table18 ALTER COLUMN "IrrelevantColumnName" DROP DEFAULT;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "schema17.table18" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE SEQUENCE "seq22";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE SEQUENCE "seq23";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   CREATE TABLE public.table10 (col10_0 DECIMAL NOT NULL, col10_1 NAME NOT NULL, col10_2 TIME NOT NULL, col10_3 OID NOT NULL, col10_4 BIT(40) NOT NULL, col10_5 REGTYPE NOT NULL, col10_6 INET NULL, col10_7 INT2 NOT NULL, col10_8 BOOL NOT NULL, col10_9 DECIMAL NULL, col10_10 REGTYPE NOT NULL, col10_11 INET NOT NULL, col10_12 OID NOT NULL, col10_13 INET NULL, col10_14 BOOL NOT NULL, col10_15 INT2 NOT NULL, col10_16 TIMESTAMPTZ NOT NULL, col10_17 REGCLASS NOT NULL, col10_18 INT8 NOT NULL, PRIMARY KEY (col10_2 DESC, col10_11 DESC, col10_17, col10_16 DESC, col10_18 ASC, col10_10 ASC, col10_14 DESC, col10_1 ASC, col10_4 DESC, col10_12 DESC, col10_5, col10_8, col10_0 ASC, col10_15 DESC, col10_7 ASC, col10_3 ASC), UNIQUE (col10_17 ASC, col10_11 ASC, col10_9 DESC, col10_1 DESC, col10_18 ASC, col10_2, col10_4 ASC, col10_14 DESC, col10_16 DESC, col10_7 ASC, col10_5 DESC, col10_12, col10_13 ASC), INDEX (col10_18 DESC, col10_4 ASC, col10_1 ASC, col10_9, col10_14 DESC, col10_12, col10_13 DESC, col10_15 DESC, col10_7 ASC, col10_6, col10_10 DESC, col10_8 ASC));
          |   SELECT 'validating all objects';
          |   ALTER TABLE public.table10 ALTER COLUMN "col10_24" SET DATA TYPE int4;
          | ROLLBACK; ERROR: column "col10_24" does not exist (SQLSTATE 42703)
          | BEGIN
          |   SELECT 'validating all objects';
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE SCHEMA IF NOT EXISTS schema21 AUTHORIZATION root;
          | ***FAIL; Expected no errors, but got ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): "sql txn" meta={id=2c77c798 key=/Table/SystemConfigSpan/Start pri=0.02241032 epo=0 ts=1606996172.343705634,1 min=1606996172.183275271,0 seq=6} lock=true stat=PENDING rts=1606996172.183275271,0 wto=false max=1606996172.183275271,0 (SQLSTATE 40001)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2491302&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) this will be resolved by https://github.com/cockroachdb/cockroach/pull/57491

(roachtest).schemachange/random-load failed on master@ce3f9b29fee565a2994ca84d2ecd20db7fe59d0b:

          | BEGIN
          |   DROP TABLE IF EXISTS public.table58;
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   ALTER TABLE public.table218 DROP COLUMN "IrrelevantColumnName";
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table218" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   SELECT 'validating all objects';
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE SEQUENCE IF NOT EXISTS public.seq63;
          |   DROP SCHEMA "schema223" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: unknown schema "schema223" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE SEQUENCE IF NOT EXISTS public.seq43;
          |   DROP SEQUENCE IF EXISTS public.seq43;
          |   ALTER TABLE public.table225 DROP COLUMN "IrrelevantColumnName";
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table225" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE TABLE IF NOT EXISTS public.table220 (col220_0 "char" NOT NULL, PRIMARY KEY (col220_0), INDEX (col220_0 DESC), UNIQUE (col220_0 ASC), UNIQUE (col220_0 ASC), INDEX (col220_0), UNIQUE (col220_0), UNIQUE (col220_0), INDEX (col220_0 DESC), UNIQUE (col220_0 ASC));
          |   ALTER TABLE public.table220 ALTER COLUMN "col220_0" SET DATA TYPE int4;
          | ROLLBACK; ERROR: ALTER COLUMN TYPE from "char" to int4 is only supported experimentally (SQLSTATE 0A000)
          | BEGIN
          |   SELECT 'validating all objects';
          |   ALTER TABLE public.table227 RENAME TO public.table228;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table227" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   SELECT 'validating all objects';
          |   SELECT 'validating all objects';
          | COMMIT; 
          | BEGIN
          |   SELECT 'validating all objects';
          |   SELECT 'validating all objects';
          |   ALTER SEQUENCE public.seq248 RENAME TO public.seq249;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.seq248" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE TABLE IF NOT EXISTS public.table224 (col224_0 VARCHAR NULL, col224_1 FLOAT8 NULL, col224_2 REGPROCEDURE NOT NULL, PRIMARY KEY (col224_2 ASC), UNIQUE (col224_2 ASC, col224_0 ASC, col224_1 DESC), UNIQUE (col224_1 DESC, col224_2 DESC, col224_0 ASC));
          |   CREATE SEQUENCE IF NOT EXISTS public.seq252;
          | Expected SQLSTATE(S) 42P01, but got no errors
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2498144&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) tracked by https://github.com/cockroachdb/cockroach/issues/57476

(roachtest).schemachange/random-load failed on master@1ed669de1c3b77798fade7ad9f056edc0bf27c36:

          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          |   0.0      0.0      0.0 txnRbk
          |    33.0s        0            0.0            3.9      0.0      0.0      0.0      0.0 opOk
          |    33.0s        0            0.0            0.9      0.0      0.0      0.0      0.0 txnCmtErr
          |    33.0s        0            0.0            0.2      0.0      0.0      0.0      0.0 txnOk
          |    33.0s        0            0.0            2.0      0.0      0.0      0.0      0.0 txnRbk
          |    34.0s        0            0.0            3.8      0.0      0.0      0.0      0.0 opOk
          |    34.0s        0            0.0            0.9      0.0      0.0      0.0      0.0 txnCmtErr
          |    34.0s        0            0.0            0.2      0.0      0.0      0.0      0.0 txnOk
          |    34.0s        0            0.0            1.9      0.0      0.0      0.0      0.0 txnRbk
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE SEQUENCE IF NOT EXISTS public.seq2881;
          |   ALTER TABLE schema3262.table3263 ALTER COLUMN "col3263_3264" SET DATA TYPE name;
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   ALTER TABLE public.table2848 ALTER COLUMN "col2848_1" DROP NOT NULL;
          | ROLLBACK; expected SQLSTATE(S) 42P16, and got ERROR: column "col2848_1" is in a primary index (SQLSTATE 42P16)
          | BEGIN
          |   CREATE INDEX IF NOT EXISTS index2848_3272 ON public.table2848 (col2848_1);
          |   ALTER TABLE public.table2848 ALTER COLUMN "col2848_1" DROP STORED;
          | ROLLBACK; ERROR: column "col2848_1" is not a computed column (SQLSTATE 42611)
          | BEGIN
          |   CREATE SCHEMA IF NOT EXISTS schema2887 AUTHORIZATION root;
          |   SELECT 'validating all objects';
          |   CREATE SCHEMA IF NOT EXISTS schema3271 AUTHORIZATION root;
          |   CREATE TABLE public.table3274 (col3274_0 DATE NOT NULL, col3274_1 "char", col3274_2 BOX2D NOT NULL, PRIMARY KEY (col3274_2 ASC, col3274_0), INDEX (col3274_1), UNIQUE (col3274_1 DESC, col3274_2 DESC, col3274_0 ASC), INDEX (col3274_0, col3274_1), UNIQUE (col3274_0 ASC, col3274_2 DESC));
          |   CREATE TYPE "public.enum3275" AS ENUM ('aaf', 'sfd', 'ssdada', 'fsaf', 'dsffaa');
          | COMMIT; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=90bb384e key=/Table/SystemConfigSpan/Start pri=0.01483271 epo=0 ts=1607341696.064509059,2 min=1607341694.932587920,0 seq=42} lock=true stat=PENDING rts=1607341694.932587920,0 wto=false max=1607341695.432587920,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE TABLE schema3.table3279 AS SELECT public.table2848.col2848_1, public.table2848.col2848_1, public.table2848.col2848_1 FROM public.table2848, public.table2848, public.table2848;
          | ROLLBACK; expected SQLSTATE(S) 42701,42712, and got ERROR: source name "table2848" specified more than once (missing AS clause) (SQLSTATE 42712)
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   ALTER TABLE public.table2848 ALTER COLUMN "col2848_1" DROP NOT NULL;
          | ROLLBACK; expected SQLSTATE(S) 42P16, and got ERROR: column "col2848_1" is in a primary index (SQLSTATE 42P16)
          | BEGIN
          |   ALTER TABLE public.table2848 RENAME CONSTRAINT "primary" TO "index2848_3273";
          |   INSERT INTO public.table2848 (col2848_1) VALUES (109025567:::OID),(2153098399:::OID);
          |   CREATE TABLE IF NOT EXISTS schema3.table3289 (col3289_0 INTERVAL NULL, col3289_1 BOX2D NOT NULL, col3289_2 REGPROC NOT NULL, col3289_3 REGTYPE NOT NULL, col3289_4 TIME NOT NULL, PRIMARY KEY (col3289_2, col3289_3 DESC, col3289_4 DESC), UNIQUE (col3289_0 DESC, col3289_3 ASC), INDEX (col3289_1, col3289_3 DESC, col3289_0 DESC), INDEX (col3289_4) STORING (col3289_1), INDEX (col3289_0) STORING (col3289_1), UNIQUE (col3289_4 DESC, col3289_0 DESC, col3289_1 DESC, col3289_3 DESC), UNIQUE (col3289_2 ASC, col3289_3 ASC), INDEX (col3289_4 ASC, col3289_2, col3289_0, col3289_1, col3289_3 ASC), INDEX (col3289_1 ASC, col3289_0 DESC, col3289_4 ASC, col3289_3, col3289_2 ASC), UNIQUE (col3289_2 DESC, col3289_4 ASC) STORING (col3289_1), FAMILY (col3289_3, col3289_1), FAMILY (col3289_0, col3289_2), FAMILY (col3289_4));
          | Expected SQLSTATE(S) 0A000, but got no errors
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2500105&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) tracked by https://github.com/cockroachdb/cockroach/issues/57476

(roachtest).schemachange/random-load failed on master@9b2aeea6ca553f79dd737052ac23c91763a0b713:

          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | RROR: column "col21138_2" of relation "table21138" already exists (SQLSTATE 42701)
          |   247.0s        0            4.0            1.2     15.7     30.4     30.4     30.4 opOk
          |   247.0s        0            0.0            0.1      0.0      0.0      0.0      0.0 txnCmtErr
          |   247.0s        0            2.0            0.1     31.5    385.9    385.9    385.9 txnOk
          |   247.0s        0            7.0            0.8      1.3      2.0      2.0      2.0 txnRbk
          | BEGIN
          |   ALTER TABLE public.table21138 RENAME COLUMN "col21138_3" TO "col21138_21156";
          |   ALTER TABLE public.table21138 RENAME COLUMN "col21138_5" TO "col21138_21164";
          |   ALTER TABLE public.table21138 RENAME CONSTRAINT "table21138_col21138_0_col21138_5_col21138_3_col21138_2_col21138_1_col21138_7_col21138_4_idx" TO "index21138_21175";
          | ROLLBACK; ERROR: constraint "table21138_col21138_0_col21138_5_col21138_3_col21138_2_col21138_1_col21138_7_col21138_4_idx" does not exist (SQLSTATE 42704)
          | BEGIN
          |   CREATE VIEW public.view21178 AS SELECT public.table21138.col21138_3, public.table21138.col21138_6, public.table21138.col21138_4, public.table21138.col21138_5, public.table21138.col21138_0, public.table21138.col21138_8, public.table21138.col21138_1 FROM public.table21138;
          |   ALTER TABLE public.table21138 ALTER COLUMN "col21138_2" DROP DEFAULT;
          |   ALTER TABLE public.table21138 DROP COLUMN "col21138_8";
          | ROLLBACK; expected SQLSTATE(S) 2BP01, and got ERROR: cannot drop column "col21138_8" because view "view21178" depends on it (SQLSTATE 2BP01)
          | BEGIN
          |   DROP INDEX public.table21138@"table21138_col21138_7_col21138_3_col21138_6_col21138_4_idx";
          | COMMIT; 
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   ALTER TABLE public.table21138 ALTER COLUMN "col21138_21205" DROP NOT NULL;
          | ROLLBACK; expected SQLSTATE(S) 42703, and got ERROR: column "col21138_21205" does not exist (SQLSTATE 42703)
          | BEGIN
          |   SELECT 'validating all objects';
          |   ALTER TABLE public.table21207 RENAME COLUMN "IrrelevantColumnName" TO "OtherIrrelevantName";
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table21207" does not exist (SQLSTATE 42P01)
          |   248.0s        0            9.0            1.2     18.9     26.2     26.2     26.2 opOk
          |   248.0s        0            0.0            0.1      0.0      0.0      0.0      0.0 txnCmtErr
          |   248.0s        0            1.0            0.1    268.4    268.4    268.4    268.4 txnOk
          |   248.0s        0            5.0            0.8      1.4      2.8      2.8      2.8 txnRbk
          | BEGIN
          |   ALTER TABLE public.table21138 ALTER COLUMN "col21138_7" DROP NOT NULL;
          |   CREATE TYPE "public.enum21215" AS ENUM ('afd');
          |   SELECT 'validating all objects';
          |   CREATE VIEW schema21227.view21228 AS SELECT public.table21138.col21138_0, public.table21138.col21138_1, public.table21138.col21138_2, public.table21138.col21138_3, public.table21138.col21138_4, public.table21138.col21138_8, public.table21138.col21138_4, public.table21138.col21138_7, public.table21138.col21138_5, public.table21138.col21138_1, public.table21138.col21138_7, public.table21138.col21138_2, public.table21138.col21138_0 FROM public.table21138, public.table21138, public.table21138;
          | ROLLBACK; expected SQLSTATE(S) 3F000,42701,42712, and got ERROR: cannot create "schema21227.view21228" because the target database or schema does not exist (SQLSTATE 3F000)
          | BEGIN
          |   ALTER TABLE public.table21138 RENAME TO public.table21231;
          |   SELECT 'validating all objects';
          |   INSERT INTO public.table21231 (col21138_0,col21138_1,col21138_2,col21138_3,col21138_4,col21138_5,col21138_6,col21138_7,col21138_8) VALUES (3436708326:::OID,1599620789:::OID,'\x935980':::BYTES,'\x723f361b':::BYTES,'}':::STRING,B'00','1981-03-15 08:42:44.000606':::TIMESTAMP,'uk':::STRING,'':::STRING:::NAME),(1962657804:::OID,4010202997:::OID,'\x':::BYTES,'\xe51c6ace6ecdaed1d2':::BYTES,'^':::STRING,B'00','2002-06-19 07:51:41.000043':::TIMESTAMP,e'\U00002603':::STRING,e'\x19CZ':::STRING:::NAME);
          |   ALTER TABLE public.table21231 RENAME COLUMN "col21138_7" TO "col21231_21245";
          | Expected SQLSTATE(S) 0A000, but got no errors
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2503533&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) tracked by https://github.com/cockroachdb/cockroach/issues/57597

(roachtest).schemachange/random-load failed on master@258ef5765d0205f487b1481a9c26059db2a70362:

test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/schemachange/random-load/run_1
    cluster.go:2214,schemachange_random_load.go:135,schemachange_random_load.go:42,test_runner.go:760: output in run_123345.057_n1_workload_run_schemachange: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2506254-1607499700-24-n3cpu4:1 -- ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 returned: exit status 20
        (1) attached stack trace
          -- stack trace:
          | main.(*cluster).RunE
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2292
          | main.(*cluster).Run
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:2212
          | main.runSchemaChangeRandomLoad
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/schemachange_random_load.go:135
          | main.registerSchemaChangeRandomLoad.func1
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/schemachange_random_load.go:42
          | main.(*testRunner).runTest.func2
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:760
          | runtime.goexit
          |     /usr/local/go/src/runtime/asm_amd64.s:1374
        Wraps: (2) output in run_123345.057_n1_workload_run_schemachange
        Wraps: (3) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2506254-1607499700-24-n3cpu4:1 -- ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 returned
          | stderr:
          | I201209 12:33:45.763792 1 workload/cli/run.go:359  creating load generator...
          | I201209 12:33:45.833721 1 workload/cli/run.go:390  creating load generator... done (took 69.929338ms)
          | Error: ***UNEXPECTED ERROR: ERROR: failed to mark job 614225889524613121 as as successful: job with status reverting cannot be marked as succeeded (SQLSTATE XXUUU)
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | 6, col836_5);
          |   ALTER TABLE schema1.table836 RENAME TO schema1.table3553;
          | COMMIT; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=b25ebe1e key=/Table/SystemConfigSpan/Start pri=0.00329101 epo=0 ts=1607517266.119671325,1 min=1607517266.077832903,0 seq=20} lock=true stat=PENDING rts=1607517266.077832903,0 wto=false max=1607517266.577832903,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER SEQUENCE public.seq3530 RENAME TO public.seq3548;
          |   DROP TABLE IF EXISTS public.table3552 RESTRICT;
          |   ALTER TABLE schema1.table836 ALTER COLUMN "col836_2" SET NOT NULL;
          |   CREATE TABLE schema3556.table3557 (col3557_0 INT2 NOT NULL, col3557_1 INT8 NOT NULL, col3557_2 BYTES NOT NULL, col3557_3 TIME NOT NULL, col3557_4 FLOAT8[] NOT NULL, col3557_5 INT4 NOT NULL, col3557_6 DECIMAL NOT NULL, col3557_7 FLOAT8 NOT NULL, col3557_8 CHAR NOT NULL, col3557_9 BOOL NOT NULL, col3557_10 VARCHAR NOT NULL, col3557_11 BIT(35) NOT NULL, col3557_12 INT2 NOT NULL, col3557_13 CHAR NULL, col3557_14 UUID NOT NULL, col3557_15 BYTES NOT NULL, PRIMARY KEY (col3557_12 ASC, col3557_15 ASC, col3557_14 ASC, col3557_9, col3557_2 DESC, col3557_10, col3557_1 ASC, col3557_6 ASC, col3557_0 DESC, col3557_11 ASC, col3557_8 DESC, col3557_5 ASC, col3557_3 ASC), UNIQUE (col3557_9 ASC, col3557_7 ASC, col3557_10, col3557_6, col3557_13 ASC, col3557_12, col3557_2 DESC, col3557_1 DESC, col3557_14 ASC, col3557_11 ASC, col3557_3 DESC, col3557_5, col3557_8, col3557_0 ASC, col3557_15 DESC), UNIQUE (col3557_13 ASC, col3557_11 ASC, col3557_5 DESC, col3557_15, col3557_0, col3557_2, col3557_1, col3557_7 DESC) STORING (col3557_4), INDEX (col3557_0, col3557_9 DESC, col3557_10 DESC, col3557_7 ASC, col3557_11 DESC, col3557_3 DESC, col3557_2 ASC, col3557_1 ASC, col3557_6 ASC, col3557_8 ASC, col3557_12 ASC, col3557_15 DESC, col3557_14 DESC), INDEX (col3557_13, col3557_2, col3557_10 ASC, col3557_12 ASC, col3557_7 DESC, col3557_11, col3557_0 ASC, col3557_9 ASC, col3557_15 ASC, col3557_3, col3557_14 ASC), UNIQUE (col3557_7 ASC, col3557_14 DESC, col3557_3 DESC, col3557_13 ASC), UNIQUE (col3557_11, col3557_12 ASC, col3557_7 ASC, col3557_0, col3557_3 ASC, col3557_10, col3557_5 DESC, col3557_15 DESC, col3557_1, col3557_14 ASC, col3557_6), INDEX (col3557_0, col3557_12 ASC, col3557_7 DESC, col3557_11, col3557_9 DESC, col3557_13 DESC, col3557_15, col3557_8 DESC, col3557_1 DESC, col3557_2 DESC, col3557_10 DESC, col3557_6 DESC, col3557_3 DESC, col3557_5 DESC, col3557_14 DESC), INDEX (col3557_11, col3557_1, col3557_7 DESC, col3557_2 DESC, col3557_13 DESC, col3557_6, col3557_8 DESC, col3557_12, col3557_9 ASC, col3557_5 DESC, col3557_10 DESC) STORING (col3557_4), INDEX (col3557_10 DESC, col3557_5, col3557_15 DESC, col3557_13 DESC, col3557_1, col3557_3 DESC, col3557_14 ASC, col3557_0 DESC, col3557_11 ASC, col3557_2 DESC, col3557_7 DESC), FAMILY (col3557_10), FAMILY (col3557_11), FAMILY (col3557_7, col3557_9, col3557_2), FAMILY (col3557_15, col3557_0, col3557_4, col3557_14, col3557_13), FAMILY (col3557_3, col3557_8), FAMILY (col3557_12, col3557_5), FAMILY (col3557_1, col3557_6));
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot create "schema3556.table3557" because the target database or schema does not exist (SQLSTATE 3F000)
          | BEGIN
          |   CREATE TABLE IF NOT EXISTS public.table3549 (col3549_0 TIMESTAMP NOT NULL, PRIMARY KEY (col3549_0 ASC), INDEX (col3549_0 ASC), UNIQUE (col3549_0), INDEX (col3549_0));
          | COMMIT; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=7fae1233 key=/Table/SystemConfigSpan/Start pri=0.00862684 epo=0 ts=1607517266.193868734,1 min=1607517266.130893576,0 seq=7} lock=true stat=PENDING rts=1607517266.130893576,0 wto=false max=1607517266.630893576,0 (SQLSTATE 40001)
          | BEGIN
          |   DROP TABLE schema1.table836;
          | ***FAIL; Expected no errors, but got ERROR: failed to mark job 614225889524613121 as as successful: job with status reverting cannot be marked as succeeded (SQLSTATE XXUUU)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2506254&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) tracked by https://github.com/cockroachdb/cockroach/issues/57494

(roachtest).schemachange/random-load failed on master@c41a2ed503cd31d95bc3ee4365663811772c8bd3:

          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/schemachange_random_load.go:135
          | main.registerSchemaChangeRandomLoad.func1
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/schemachange_random_load.go:42
          | main.(*testRunner).runTest.func2
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:760
          | runtime.goexit
          |     /usr/local/go/src/runtime/asm_amd64.s:1374
        Wraps: (2) output in run_115612.145_n1_workload_run_schemachange
        Wraps: (3) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2509111-1607583926-22-n3cpu4:1 -- ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 returned
          | stderr:
          | I201210 11:56:12.889407 1 workload/cli/run.go:359  creating load generator...
          | I201210 11:56:12.963938 1 workload/cli/run.go:390  creating load generator... done (took 74.530607ms)
          | Error: ***UNEXPECTED SUCCESS
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          |   0            0.0            4.0      0.0      0.0      0.0      0.0 txnRbk
          |    39.0s        0            0.0            5.9      0.0      0.0      0.0      0.0 opOk
          |    39.0s        0            0.0            0.9      0.0      0.0      0.0      0.0 txnCmtErr
          |    39.0s        0            0.0            0.4      0.0      0.0      0.0      0.0 txnOk
          |    39.0s        0            0.0            3.9      0.0      0.0      0.0      0.0 txnRbk
          | BEGIN
          |   ALTER TABLE schema2826.table2827 ALTER COLUMN "col2827_2828" SET DATA TYPE timestamp;
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | BEGIN
          |   ALTER TABLE schema26.table2442 ALTER COLUMN "col2442_8" DROP STORED;
          | ROLLBACK; ERROR: column "col2442_8" is not a computed column (SQLSTATE 42611)
          | BEGIN
          |   CREATE INDEX index2442_2837 ON schema26.table2442 (col2442_2 ASC, col2442_7 DESC, col2442_0, col2442_10, col2442_12, col2442_9 DESC, col2442_8 ASC, col2442_13 DESC) STORING (col2442_3, col2442_4, col2442_2564, col2442_6, col2442_15, col2442_5, col2442_14);
          | ROLLBACK; ERROR: unimplemented: column col2442_8 is of type bit[] and thus is not indexable (SQLSTATE 0A000)
          | BEGIN
          |   CREATE TABLE schema26.table2844 AS SELECT public.table1767.col270_5, public.table1767.col270_6, public.table1767.col270_4, schema26.table2442.col2442_15, schema26.table2442.col2442_4, schema26.table2442.col2442_14, schema26.table2442.col2442_9, schema26.table2442.col2442_3, schema26.table2442.col2442_2564, schema26.table2442.col2442_13, schema26.table2442.col2442_2, schema26.table2442.col2442_6, schema26.table2442.col2442_10, schema26.table2442.col2442_12, public.table1767.col270_6, public.table1767.col270_1, public.table1767.col270_4 FROM public.table1767, schema26.table2442, public.table1767;
          | ROLLBACK; expected SQLSTATE(S) 42701,42712, and got ERROR: source name "table1767" specified more than once (missing AS clause) (SQLSTATE 42712)
          | BEGIN
          |   INSERT INTO schema26.table2442 (col2442_0,col2442_1,col2442_2,col2442_3,col2442_4,col2442_5,col2442_6,col2442_7,col2442_8,col2442_9,col2442_10,col2442_11,col2442_12,col2442_13,col2442_14,col2442_15,col2442_2564) VALUES ('1970-08-21 04:48:53.000265':::TIMESTAMP,'15:15:04.508131':::TIME,0.07938247027194445:::FLOAT8,e'aNu\x1bLa+':::STRING:::NAME,'\x9a889cde96':::BYTES,116626077:::OID,3661349696:::OID,'':::STRING,ARRAY[B'111000101001000011011101011100100011100000',B'000000000000000000000000000000000000000000',B'010111100110000001100011011110001011101100',B'010100010110011010100101011010011101110010'],'20:26:26.339998':::TIME,3056969770:::OID,0:::OID,'-':::STRING,'2015-09-26 16:21:09.000995':::TIMESTAMP,5.170102469556544838E+21:::DECIMAL,'01:19:04.429429+02:35:00':::TIMETZ,B'00001'),('1983-02-01 19:50:35.000436':::TIMESTAMP,'10:33:41.791093':::TIME,1.243203080792383:::FLOAT8,'9':::STRING:::NAME,'\x56bb4a0fb514702e7c':::BYTES,3121013721:::OID,205948291:::OID,NULL,ARRAY[B'000001100010101010000111100000011111111101',B'111010010110000001111100011100011100100110',B'110010110111110000100011101011010001100000'],'14:58:13.986724':::TIME,2090090002:::OID,0:::OID,e'\x02':::STRING,'2021-06-25 05:51:35.00084':::TIMESTAMP,(-18237.74597411796744):::DECIMAL,'09:04:48.246792-11:12:00':::TIMETZ,B'10100');
          |   ALTER TABLE schema26.table2442 ALTER COLUMN "col2442_10" SET DATA TYPE float4;
          | ROLLBACK; ERROR: unimplemented: schema change statement cannot follow a statement that has written in the same transaction (SQLSTATE 0A000)
          | BEGIN
          |   INSERT INTO public.table1767 (col270_0,col270_1,col270_2,col270_3,col270_4,col270_5,col270_6) VALUES (B'010',2.5293564796447754:::FLOAT8,540024352:::OID,(-0.802123510465963):::FLOAT8,'a54b02d4-f717-473a-b5fa-00de1b2738dd':::UUID,(-1070718430):::INT8,1495775128:::OID),(B'000','-Inf':::FLOAT8,579181219:::OID,(-0.11649037138237378):::FLOAT8,'fb3ac710-4717-49b2-932e-10e2a6e338b8':::UUID,NULL,4149123582:::OID),(B'000',1.401298464324817e-45:::FLOAT8,909446394:::OID,0.7175622103881709:::FLOAT8,'380c4b45-6dae-4a42-a99c-7340e5ef4097':::UUID,1360871919:::INT8,2477618421:::OID);
          | Expected SQLSTATE(S) 42P01, but got no errors
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2509111&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) tracked by https://github.com/cockroachdb/cockroach/issues/57476

(roachtest).schemachange/random-load failed on master@960b4cfc54c8b78df56e62f07d7a07b986ceacff:

          | I201212 11:34:40.449151 1 workload/cli/run.go:359  creating load generator...
          | I201212 11:34:40.510841 1 workload/cli/run.go:390  creating load generator... done (took 61.688394ms)
          | Error: ***UNEXPECTED SUCCESS
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | ot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE SEQUENCE IF NOT EXISTS public.seq839;
          |   ALTER SEQUENCE public.seq839 RENAME TO public.seq842;
          |   SELECT 'validating all objects';
          |   SELECT 'validating all objects';
          | COMMIT; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=c25599a9 key=/Table/SystemConfigSpan/Start pri=0.04444732 epo=0 ts=1607772887.434292090,2 min=1607772880.570994001,0 seq=22} lock=true stat=PENDING rts=1607772880.570994001,0 wto=false max=1607772881.070994001,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE TABLE schema3.table854 (col854_0 FLOAT4 NOT NULL, col854_1 BIT(6) NOT NULL, col854_2 BYTES NULL, UNIQUE (col854_0 DESC, col854_1 ASC), INDEX (col854_0, col854_1) STORING (col854_2), INDEX (col854_1 DESC) STORING (col854_0, col854_2), INDEX (col854_2 ASC), INDEX (col854_2 ASC, col854_0, col854_1 DESC), UNIQUE (col854_1) STORING (col854_0, col854_2), INDEX (col854_0 ASC, col854_2 DESC), UNIQUE (col854_0 ASC) STORING (col854_2));
          |   DROP INDEX schema3.table854@"table854_col854_2_idx";
          |   ALTER TABLE schema3.table854 RENAME COLUMN "col854_2" TO "col854_863";
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_ABORT_SPAN): "sql txn" meta={id=6147a498 key=/Table/SystemConfigSpan/Start pri=0.04224260 epo=0 ts=1607772887.434292090,2 min=1607772880.835320120,0 seq=22} lock=true stat=ABORTED rts=1607772880.835320120,0 wto=false max=1607772880.835320120,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE TYPE "public.enum837" AS ENUM ('as', 'sfddd', 'fsff');
          |   CREATE TYPE "schema3.enum844" AS ENUM ('adfas');
          |   CREATE SCHEMA schema855 AUTHORIZATION root;
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_PUSHER_ABORTED): "sql txn" meta={id=d9a81568 key=/Table/SystemConfigSpan/Start pri=0.01575910 epo=0 ts=1607772887.434292090,2 min=1607772880.631348612,0 seq=23} lock=true stat=ABORTED rts=1607772880.631348612,0 wto=false max=1607772880.631348612,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE TYPE "public.enum853" AS ENUM ('adaaff', 'f');
          |   CREATE SEQUENCE IF NOT EXISTS schema3.seq859;
          |   CREATE SEQUENCE IF NOT EXISTS public.seq862;
          |   ALTER SEQUENCE schema3.seq859 RENAME TO schema3.seq865;
          | COMMIT; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=37188af2 key=/Table/SystemConfigSpan/Start pri=0.06563328 epo=0 ts=1607772888.988785652,1 min=1607772880.849412994,0 seq=39} lock=true stat=PENDING rts=1607772880.849412994,0 wto=false max=1607772881.349412994,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE TABLE schema3.table860 (col860_0 DATE NOT NULL, col860_1 FLOAT8, col860_2 BIT(30)[] NULL, col860_3 TIMETZ NOT NULL, col860_4 BOX2D NOT NULL, col860_5 INT2 NOT NULL, col860_6 INET, col860_7 VARCHAR NULL, col860_8 TIMETZ NOT NULL, col860_9 INT4 NOT NULL, PRIMARY KEY (col860_8), UNIQUE (col860_8 DESC), FAMILY (col860_2), FAMILY (col860_5, col860_8, col860_9), FAMILY (col860_4, col860_0, col860_3, col860_6), FAMILY (col860_1, col860_7));
          |   INSERT INTO schema3.table860 (col860_0,col860_1,col860_2,col860_3,col860_4,col860_5,col860_6,col860_7,col860_8,col860_9) VALUES ('1982-11-30':::DATE,(-0.5041707444513115):::FLOAT8,ARRAY[B'010010011111101101010111001101',B'011101100110110110001010010000',B'000111001000011001100110000110'],'10:12:14.583387-09:45:00':::TIMETZ,'BOX(-0.12856949172938203 -0.27702081097144726,0.0411633236246397 0.6441299790382503)':::BOX2D,2221:::INT8,'7bef:e698:7d8c:7ef4:233f:c83d:2a3e:f6cd/34':::INET,e'#:\x1c!Vx':::STRING,'24:00:00-15:59:00':::TIMETZ,(-519967220):::INT8);
          |   ALTER TABLE schema3.table860 DROP COLUMN "col860_6";
          | Expected SQLSTATE(S) 0A000, but got no errors
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2515122&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(roachtest).schemachange/random-load failed on master@53b08837cf5e76504e437bad80c97e05989d2c60:

          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | 0      0.0 txnOk
          |   145.0s        0            0.0            1.1      0.0      0.0      0.0      0.0 txnRbk
          | _elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
          |   146.0s        0            0.0            1.2      0.0      0.0      0.0      0.0 opOk
          |   146.0s        0            0.0            0.1      0.0      0.0      0.0      0.0 txnCmtErr
          |   146.0s        0            0.0            0.2      0.0      0.0      0.0      0.0 txnOk
          |   146.0s        0            0.0            1.1      0.0      0.0      0.0      0.0 txnRbk
          |   147.0s        0            0.0            1.2      0.0      0.0      0.0      0.0 opOk
          |   147.0s        0            0.0            0.1      0.0      0.0      0.0      0.0 txnCmtErr
          |   147.0s        0            0.0            0.2      0.0      0.0      0.0      0.0 txnOk
          |   147.0s        0            0.0            1.1      0.0      0.0      0.0      0.0 txnRbk
          |   148.0s        0            0.0            1.2      0.0      0.0      0.0      0.0 opOk
          |   148.0s        0            0.0            0.1      0.0      0.0      0.0      0.0 txnCmtErr
          |   148.0s        0            0.0            0.2      0.0      0.0      0.0      0.0 txnOk
          |   148.0s        0            0.0            1.1      0.0      0.0      0.0      0.0 txnRbk
          |   149.0s        0            0.0            1.2      0.0      0.0      0.0      0.0 opOk
          |   149.0s        0            0.0            0.1      0.0      0.0      0.0      0.0 txnCmtErr
          |   149.0s        0            0.0            0.2      0.0      0.0      0.0      0.0 txnOk
          |   149.0s        0            0.0            1.1      0.0      0.0      0.0      0.0 txnRbk
          |   150.0s        0            0.0            1.1      0.0      0.0      0.0      0.0 opOk
          |   150.0s        0            0.0            0.1      0.0      0.0      0.0      0.0 txnCmtErr
          |   150.0s        0            0.0            0.2      0.0      0.0      0.0      0.0 txnOk
          |   150.0s        0            0.0            1.1      0.0      0.0      0.0      0.0 txnRbk
          | BEGIN
          |   ALTER TABLE schema12277.table12278 RENAME CONSTRAINT "index12278_12279" TO "index12278_12280";
          | ROLLBACK; ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02)
          | _elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
          |   151.0s        0            0.0            1.1      0.0      0.0      0.0      0.0 opOk
          |   151.0s        0            0.0            0.1      0.0      0.0      0.0      0.0 txnCmtErr
          |   151.0s        0            0.0            0.2      0.0      0.0      0.0      0.0 txnOk
          |   151.0s        0            1.0            1.1      0.2      0.2      0.2      0.2 txnRbk
          |   152.0s        0            0.0            1.1      0.0      0.0      0.0      0.0 opOk
          |   152.0s        0            0.0            0.1      0.0      0.0      0.0      0.0 txnCmtErr
          |   152.0s        0            0.0            0.2      0.0      0.0      0.0      0.0 txnOk
          |   152.0s        0            0.0            1.1      0.0      0.0      0.0      0.0 txnRbk
          | BEGIN
          |   ALTER TABLE schema57.table11628 ALTER COLUMN "col10332_6" SET DATA TYPE interval;
          | ROLLBACK; ERROR: cannot convert TIMETZ to INTERVAL (SQLSTATE 42846)
          | BEGIN
          |   ALTER TABLE schema57.table11628 RENAME TO public.table12408;
          | ROLLBACK; expected SQLSTATE(S) 42602, and got ERROR: cannot change schema of table with RENAME (SQLSTATE 42602)
          | BEGIN
          |   INSERT INTO schema57.table11628 (col10332_0,col10332_1,col10332_2,col10332_4,col10332_5,col10332_6,col10332_7,col10332_8) VALUES ('J':::STRING,2903527831:::OID,ARRAY['\xa3e76f':::BYTES,'\xc5d759c098ee':::BYTES],'-3 years -727 days -20:29:08.627389':::INTERVAL,3973917734:::OID,'07:10:27.278758+01:28:00':::TIMETZ,'231b12fe-2cce-463b-a5a2-bcd59b0ab623':::UUID,'#':::STRING),('':::STRING,132404388:::OID,ARRAY['\x00':::BYTES,'\x72':::BYTES,'\x28454db477':::BYTES],'39 years 3 mons 995 days 19:31:52.278406':::INTERVAL,3277358042:::OID,NULL,'dfef670c-82fb-4ed0-a9e2-df5c7bb31793':::UUID,'H':::STRING);
          | ***FAIL; Expected no errors, but got ERROR: bit string length 0 does not match type BIT(28) (SQLSTATE 22026)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2516757&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) tracked by https://github.com/cockroachdb/cockroach/issues/57494

(roachtest).schemachange/random-load failed on master@3018f66208f68fb90c77e3fad01f395a8f10ca8b:

          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | ABLE schema8050.table10846 RENAME TO schema8050.table13090;
          |   ALTER TABLE schema8050.table13090 ALTER COLUMN "col10846_5" DROP NOT NULL;
          | ROLLBACK; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_ABORT_SPAN): "sql txn" meta={id=df65edc6 key=/Table/SystemConfigSpan/Start pri=0.02786424 epo=0 ts=1608035952.033331753,1 min=1608035946.582315778,0 seq=12} lock=true stat=ABORTED rts=1608035946.582315778,0 wto=false max=1608035946.582315778,0 (SQLSTATE 40001)
          | _elapsed___errors__ops/sec(inst)___ops/sec(cum)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)
          |   176.0s        0            9.0            1.5     29.4     96.5     96.5     96.5 opOk
          |   176.0s        0            2.0            0.3   5905.6   6174.0   6174.0   6174.0 txnCmtErr
          |   176.0s        0            0.0            0.1      0.0      0.0      0.0      0.0 txnOk
          |   176.0s        0            9.0            1.1      3.3     11.0     11.0     11.0 txnRbk
          | BEGIN
          |   CREATE TABLE schema8050.table13079 (col13079_0 FLOAT8 NULL, col13079_1 REGTYPE, col13079_2 TIMESTAMPTZ NOT NULL, col13079_3 STRING NOT NULL, col13079_4 STRING NOT NULL, PRIMARY KEY (col13079_3 ASC, col13079_4, col13079_2 DESC), UNIQUE (col13079_0, col13079_2 ASC, col13079_1, col13079_3 DESC, col13079_4 ASC), UNIQUE (col13079_3 DESC, col13079_2 DESC, col13079_1 ASC, col13079_0 ASC, col13079_4 ASC), INDEX (col13079_3 DESC, col13079_2 ASC, col13079_4, col13079_0 DESC) STORING (col13079_1), UNIQUE (col13079_0, col13079_2 DESC, col13079_1), UNIQUE (col13079_4 ASC, col13079_2 ASC, col13079_3 ASC, col13079_1 ASC, col13079_0), INDEX (col13079_2, col13079_4, col13079_1 DESC));
          | COMMIT; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=5c396654 key=/Table/SystemConfigSpan/Start pri=0.01639635 epo=0 ts=1608035952.356129252,1 min=1608035952.100417657,0 seq=7} lock=true stat=PENDING rts=1608035952.100417657,0 wto=false max=1608035952.600417657,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          |   ALTER TABLE schema8050.table13108 ADD COLUMN IrrelevantColumnName string;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "schema8050.table13108" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE INDEX index10846_13109 ON schema8050.table10846 (col10846_13 ASC, col10846_11 ASC) STORING (col10846_14, col10846_0, col10846_1, col10846_12, col10846_9, col10846_5, col10846_7, col10846_6, col10846_15, col10846_2);
          | ROLLBACK; ERROR: index "index10846_13109" already contains column "col10846_12" (SQLSTATE 42701)
          | BEGIN
          |   ALTER TABLE schema8050.table10846 ALTER COLUMN "col10846_11" DROP STORED;
          | ROLLBACK; ERROR: column "col10846_11" is not a computed column (SQLSTATE 42611)
          | BEGIN
          |   ALTER TABLE schema8050.table10846 DROP COLUMN "col10846_4";
          | ROLLBACK; expected SQLSTATE(S) 42P10, and got ERROR: column "col10846_4" is referenced by the primary key (SQLSTATE 42P10)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE INDEX IF NOT EXISTS index10846_13112 ON schema8050.table10846 (col10846_15 ASC, col10846_14 ASC, col10846_8 ASC, col10846_12 DESC, col10846_2, col10846_0 ASC, col10846_13 ASC, col10846_4, col10846_11 ASC) STORING (col10846_5, col10846_1, col10846_10, col10846_3, col10846_7, col10846_9, col10846_6);
          | ROLLBACK; ERROR: index "index10846_13112" already contains column "col10846_7" (SQLSTATE 42701)
          | BEGIN
          |   ALTER TABLE schema8050.table10846 ALTER COLUMN "col10846_6" SET DATA TYPE oid[];
          | ROLLBACK; ERROR: cannot convert OID to OID[] (SQLSTATE 42846)
          | BEGIN
          |   CREATE TABLE IF NOT EXISTS schema8050.table13091 (col13091_0 OID NOT NULL, col13091_1 REGPROC NOT NULL, col13091_2 REGPROC NULL, col13091_3 BYTES, PRIMARY KEY (col13091_1 ASC), UNIQUE (col13091_1) STORING (col13091_2, col13091_3), INDEX (col13091_2, col13091_0 ASC));
          |   SELECT 'validating all objects';
          |   ALTER TABLE schema8050.table13091 RENAME TO schema8050.table13115;
          |   DROP TABLE schema8050.table13091 CASCADE;
          | ***FAIL; Expected no errors, but got ERROR: relation "schema8050.table13091" does not exist (SQLSTATE 42P01)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2519773&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) tracked by https://github.com/cockroachdb/cockroach/issues/57476

(roachtest).schemachange/random-load failed on master@de91557c0634ab3797356d445ba39e37d45d8205:

        Wraps: (2) output in run_123834.395_n1_workload_run_schemachange
        Wraps: (3) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2522960-1608102201-68-n3cpu4:1 -- ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 returned
          | stderr:
          | I201216 12:38:35.137968 1 workload/cli/run.go:359  creating load generator...
          | I201216 12:38:35.226341 1 workload/cli/run.go:390  creating load generator... done (took 88.371814ms)
          | Error: ***UNEXPECTED SUCCESS
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | ransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=9abfbc5d key=/Table/SystemConfigSpan/Start pri=0.12934008 epo=0 ts=1608122315.277745119,1 min=1608122315.230070034,0 seq=8} lock=true stat=PENDING rts=1608122315.230070034,0 wto=false max=1608122315.730070034,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER SEQUENCE public.seq15 RENAME TO public.seq16;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.seq15" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   SELECT 'validating all objects';
          |   ALTER INDEX public.table17@"IrrelevantConstraintName" RENAME TO "OtherConstraintName";
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table17" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE TYPE public.enum3 AS ENUM ('ssd', 'ss', 'ada', 'af', 'afsdfa');
          |   ALTER TABLE public.table18 DROP CONSTRAINT IrrelevantConstraintName;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table18" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   ALTER TABLE public.table12 RENAME TO public.table13;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table12" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE SEQUENCE IF NOT EXISTS public.seq21;
          | COMMIT; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=bb60406f key=/Table/SystemConfigSpan/Start pri=0.00502481 epo=0 ts=1608122315.277745119,1 min=1608122315.227552789,0 seq=8} lock=true stat=PENDING rts=1608122315.227552789,0 wto=false max=1608122315.727552789,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE SEQUENCE IF NOT EXISTS schema23.seq24;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot create "schema23.seq24" because the target database or schema does not exist (SQLSTATE 3F000)
          | BEGIN
          |   CREATE TABLE IF NOT EXISTS public.table2 (col2_0 DECIMAL NOT NULL, col2_1 TIME NOT NULL, col2_2 NAME, col2_3 DECIMAL NOT NULL, col2_4 INET NOT NULL, col2_5 TIMESTAMPTZ NOT NULL, col2_6 INT4 NULL, col2_7 REGPROCEDURE, col2_8 TIME NOT NULL, col2_9 DATE NULL, col2_10 DATE, col2_11 TIMESTAMPTZ NOT NULL, col2_12 TIMESTAMP NOT NULL, col2_13 NAME NOT NULL, PRIMARY KEY (col2_11 DESC, col2_1, col2_12 ASC, col2_5 DESC, col2_4 ASC, col2_3), UNIQUE (col2_13, col2_11 ASC, col2_5 DESC, col2_2 ASC, col2_7 ASC, col2_3, col2_8, col2_12 DESC, col2_0 DESC, col2_1 DESC, col2_10 ASC), UNIQUE (col2_10 DESC, col2_12 ASC, col2_11), UNIQUE (col2_8 ASC, col2_0 DESC, col2_13 ASC, col2_3 ASC, col2_1 ASC, col2_9 ASC, col2_6 DESC, col2_7, col2_10, col2_2), INDEX (col2_0, col2_3, col2_6 ASC, col2_7, col2_11), INDEX (col2_2, col2_3 ASC, col2_6, col2_9, col2_11, col2_5 ASC, col2_1 DESC, col2_4 ASC, col2_7, col2_8 ASC, col2_13, col2_12, col2_10, col2_0 ASC), INDEX (col2_5) STORING (col2_0, col2_2, col2_7));
          |   CREATE TABLE public.table19 AS SELECT public.table2.col2_0, public.table2.col2_10, public.table2.col2_13, public.table2.col2_4, public.table2.col2_2, public.table2.col2_9, public.table2.col2_8, public.table2.col2_6, public.table2.col2_12, public.table2.col2_3, public.table2.col2_1, public.table2.col2_5, public.table2.col2_7 FROM public.table2;
          |   CREATE TYPE public.enum22 AS ENUM ('ssdfd', 'aadsfa', 'sfffaf', 'dsf');
          | COMMIT; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=d3d0eeab key=/Table/SystemConfigSpan/Start pri=0.00709491 epo=0 ts=1608122315.277745119,1 min=1608122315.228435153,0 seq=23} lock=true stat=PENDING rts=1608122315.228435153,0 wto=false max=1608122315.728435153,0 (SQLSTATE 40001)
          | BEGIN
          |   CREATE TABLE public.table8 (col8_0 TIMESTAMP NOT NULL, col8_1 INT8 NOT NULL, PRIMARY KEY (col8_0 DESC), INDEX (col8_1 ASC), INDEX (col8_1 ASC), INDEX (col8_0 ASC) STORING (col8_1));
          |   INSERT INTO public.table8 (col8_0,col8_1) VALUES ('2017-05-17 01:02:40.000475':::TIMESTAMP,7432668822644946161:::INT8);
          |   DROP TABLE public.table8 RESTRICT;
          | Expected SQLSTATE(S) 0A000, but got no errors
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2522960&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) tracked by https://github.com/cockroachdb/cockroach/issues/57476

(roachtest).schemachange/random-load failed on master@eda9189cecbbc279f1857f6e6b992bdfd363397e:

          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/schemachange_random_load.go:42
          | main.(*testRunner).runTest.func2
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:760
          | runtime.goexit
          |     /usr/local/go/src/runtime/asm_amd64.s:1374
        Wraps: (2) output in run_120249.543_n1_workload_run_schemachange
        Wraps: (3) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2525484-1608188837-71-n3cpu4:1 -- ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 returned
          | stderr:
          | I201217 12:02:50.278524 1 workload/cli/run.go:359  creating load generator...
          | I201217 12:02:50.354250 1 workload/cli/run.go:390  creating load generator... done (took 75.724916ms)
          | Error: ***UNEXPECTED SUCCESS
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | flag converted to WriteTooOldError): "sql txn" meta={id=3cadf6fa key=/Table/SystemConfigSpan/Start pri=0.01247297 epo=0 ts=1608206570.607710133,1 min=1608206570.355830338,0 seq=14} lock=true stat=PENDING rts=1608206570.355830338,0 wto=false max=1608206570.355830338,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER TABLE public.table18 ALTER COLUMN IrrelevantColumnName SET DEFAULT "IrrelevantValue";
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table18" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE TYPE public.enum16 AS ENUM ('fd', 'ffsdfs', 'sad', 'ss', 'dddas', 'sdaf');
          | COMMIT; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=0b9f1208 key=/Table/SystemConfigSpan/Start pri=0.00535836 epo=0 ts=1608206570.356800533,1 min=1608206570.354891440,0 seq=9} lock=true stat=PENDING rts=1608206570.354891440,0 wto=false max=1608206570.854891440,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          |   ALTER TABLE public.table21 RENAME TO public.table22;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table21" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   SELECT 'validating all objects';
          |   SELECT 'validating all objects';
          |   SELECT 'validating all objects';
          |   SELECT 'validating all objects';
          | COMMIT; 
          | BEGIN
          |   DROP SEQUENCE IF EXISTS public.seq7;
          |   CREATE TABLE public.table14 (col14_0 TIMETZ NOT NULL, col14_1 FLOAT4 NOT NULL, col14_2 TIME NOT NULL, col14_3 REGTYPE NOT NULL, col14_4 "char" NOT NULL, col14_5 UUID NOT NULL, col14_6 INTERVAL NOT NULL, col14_7 DATE NOT NULL, col14_8 VARCHAR NOT NULL, col14_9 FLOAT8 NOT NULL, col14_10 REGNAMESPACE NOT NULL, col14_11 "char" NOT NULL, col14_12 INET NOT NULL, col14_13 INTERVAL NOT NULL, col14_14 OID NOT NULL, col14_15 BYTES NOT NULL, col14_16 INTERVAL NOT NULL, PRIMARY KEY (col14_16 ASC, col14_0 DESC, col14_14 ASC, col14_9, col14_8, col14_11 ASC, col14_13, col14_15 DESC, col14_2, col14_3 ASC, col14_12 DESC, col14_7 DESC, col14_5 DESC, col14_4 DESC, col14_6 DESC, col14_1 ASC, col14_10 ASC), UNIQUE (col14_10, col14_8 DESC, col14_0, col14_15, col14_4 DESC, col14_9 ASC, col14_7 DESC, col14_3, col14_2 ASC, col14_5, col14_12 DESC, col14_6, col14_16 DESC, col14_14 ASC, col14_11), UNIQUE (col14_4, col14_0 ASC, col14_9, col14_5 ASC, col14_12 ASC, col14_7 ASC, col14_10 DESC, col14_8, col14_13 DESC, col14_14 DESC, col14_15 DESC, col14_3 ASC, col14_6 DESC, col14_1 DESC, col14_2 DESC), UNIQUE (col14_3 ASC, col14_11 ASC, col14_7, col14_14 DESC, col14_16, col14_1, col14_6 ASC, col14_12 DESC, col14_9, col14_13 DESC), UNIQUE (col14_15 ASC, col14_0, col14_3 DESC, col14_1 ASC, col14_5, col14_6, col14_8 ASC, col14_13 DESC, col14_12 DESC, col14_2 ASC, col14_14 ASC), UNIQUE (col14_6 ASC), INDEX (col14_12 ASC, col14_16, col14_2 DESC, col14_7 ASC, col14_1 ASC, col14_9, col14_14 DESC, col14_4, col14_0, col14_8), FAMILY (col14_14), FAMILY (col14_1), FAMILY (col14_9), FAMILY (col14_12), FAMILY (col14_8), FAMILY (col14_10), FAMILY (col14_5, col14_16), FAMILY (col14_3, col14_6, col14_0), FAMILY (col14_2), FAMILY (col14_15), FAMILY (col14_11), FAMILY (col14_13, col14_4), FAMILY (col14_7));
          |   INSERT INTO public.table14 (col14_0,col14_1,col14_2,col14_3,col14_4,col14_5,col14_6,col14_7,col14_8,col14_9,col14_10,col14_11,col14_12,col14_13,col14_14,col14_15,col14_16) VALUES ('18:12:18.050952+13:39:00':::TIMETZ,(-0.3776683807373047):::FLOAT8,'16:38:29.871901':::TIME,2777449313:::OID,'e':::STRING,'724d72e3-61b8-4c69-bae2-bc8bc09f18de':::UUID,'-58 years -3 mons -714 days -20:59:39.353291':::INTERVAL,'1971-05-05':::DATE,e'Ep\x07:P6':::STRING,(-0.08601441867813789):::FLOAT8,1323167563:::OID,'?':::STRING,'77.110.127.228/15':::INET,'-51 years -3 mons -927 days -09:20:33.68157':::INTERVAL,2265416657:::OID,'\x9693a0ac0e6480f6':::BYTES,'12 years 1 mon 358 days 00:42:23.634308':::INTERVAL);
          |   DROP TABLE public.table14;
          | Expected SQLSTATE(S) 0A000, but got no errors
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2525484&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) will be resolved by https://github.com/cockroachdb/cockroach/pull/58065

(roachtest).schemachange/random-load failed on master@46919380225dba7122130c338744b561d7eb6c56:

          | main.(*testRunner).runTest.func2
          |     /home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:760
          | runtime.goexit
          |     /usr/local/go/src/runtime/asm_amd64.s:1374
        Wraps: (2) output in run_115717.195_n1_workload_run_schemachange
        Wraps: (3) /home/agent/work/.go/src/github.com/cockroachdb/cockroach/bin/roachprod run teamcity-2528188-1608274662-71-n3cpu4:1 -- ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20 returned
          | stderr:
          | I201218 11:57:17.951437 1 workload/cli/run.go:359  creating load generator...
          | I201218 11:57:18.011072 1 workload/cli/run.go:390  creating load generator... done (took 59.63447ms)
          | Error: ***UNEXPECTED ERROR: ERROR: relation "public.table25" does not exist (SQLSTATE 42P01)
          | Error: COMMAND_PROBLEM: exit status 1
          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   SELECT 'validating all objects';
          | COMMIT; 
          | BEGIN
          |   ALTER TABLE public.table5 ALTER COLUMN IrrelevantColumnName SET DATA TYPE IrrelevantDataType;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table5" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   ALTER VIEW public.view3 RENAME TO public.view4;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.view3" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE TYPE public.enum7 AS ENUM ('fs', 'sfddfs', 'sdd', 'fasda', 'sfad');
          | COMMIT; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=c6f7f763 key=/Table/SystemConfigSpan/Start pri=0.01481161 epo=0 ts=1608292638.014030127,1 min=1608292638.013351547,0 seq=9} lock=true stat=PENDING rts=1608292638.013351547,0 wto=false max=1608292638.513351547,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER TABLE public.table20 ALTER COLUMN IrrelevantColumnName SET NOT NULL;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table20" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE SEQUENCE IF NOT EXISTS public.seq6;
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE SEQUENCE IF NOT EXISTS public.seq24 OWNED BY public.table25."IrrelevantColumnName";
          | ***FAIL; Expected no errors, but got ERROR: relation "public.table25" does not exist (SQLSTATE 42P01)
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2528188&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) tracked by https://github.com/cockroachdb/cockroach/issues/58068

(roachtest).schemachange/random-load failed on master@a782ee8d93a23fc53eedada3c51893c19f7bb41e:

          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | ETZ NOT NULL, col16_29 DECIMAL, UNIQUE (col16_27 ASC, col16_29, col16_18 ASC, col16_26 ASC, col16_22 ASC) STORING (col16_17, col16_19, col16_21, col16_23, col16_24, col16_28), UNIQUE (col16_22 DESC, col16_19 ASC, col16_23 DESC, col16_29, col16_27 ASC, col16_28 ASC, col16_26, col16_18 ASC, col16_17) STORING (col16_20, col16_25), UNIQUE (col16_17, col16_25, col16_21 ASC, col16_19 DESC, col16_29 DESC, col16_20, col16_23 ASC, col16_26 DESC, col16_27 ASC, col16_28 ASC, col16_18 DESC) STORING (col16_22, col16_24), FAMILY (col16_19, col16_26), FAMILY (col16_23), FAMILY (col16_25), FAMILY (col16_22), FAMILY (col16_18, col16_24), FAMILY (col16_27, col16_21, col16_17), FAMILY (col16_28), FAMILY (col16_20, col16_29));
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot create "schema15.table16" because the target database or schema does not exist (SQLSTATE 3F000)
          | BEGIN
          |   CREATE TABLE IF NOT EXISTS public.table2 (col2_3 VARCHAR NOT NULL, col2_4 BOX2D NULL, col2_5 FLOAT4, col2_6 TIMETZ[], col2_7 CHAR NOT NULL, PRIMARY KEY (col2_7, col2_3));
          |   SELECT 'validating all objects';
          |   ALTER TABLE public.table2 ALTER COLUMN "col2_7" DROP NOT NULL;
          | ROLLBACK; expected SQLSTATE(S) 42P16, and got ERROR: column "col2_7" is in a primary index (SQLSTATE 42P16)
          | BEGIN
          |   ALTER TABLE public.table45 ALTER COLUMN IrrelevantColumnName SET DATA TYPE IrrelevantDataType;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table45" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE SEQUENCE IF NOT EXISTS public.seq34;
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   CREATE TYPE public.enum11 AS ENUM ('aafdfs');
          |   ALTER TABLE public.table47 ALTER COLUMN IrrelevantColumnName DROP STORED;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "public.table47" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE TABLE IF NOT EXISTS public.table37 (col37_38 TIMESTAMPTZ, col37_39 BIT NOT NULL, col37_40 NAME NOT NULL, col37_41 TIMESTAMP NULL, col37_42 TIME, col37_43 BOOL[], col37_44 REGTYPE NOT NULL, PRIMARY KEY (col37_40 ASC), FAMILY (col37_40), FAMILY (col37_43), FAMILY (col37_38, col37_42), FAMILY (col37_44), FAMILY (col37_39), FAMILY (col37_41));
          |   ALTER TABLE public.table37 DROP COLUMN "col37_44";
          | COMMIT; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=4fb8d475 key=/Table/SystemConfigSpan/Start pri=0.02459688 epo=0 ts=1608379271.029693191,1 min=1608379270.902362035,0 seq=19} lock=true stat=PENDING rts=1608379270.902362035,0 wto=false max=1608379271.402362035,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE TYPE public.enum31 AS ENUM ('ddf', 'dasass', 's', 'aass', 'fssds');
          |   SELECT 'validating all objects';
          | COMMIT; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=c968b63a key=/Table/SystemConfigSpan/Start pri=0.01838030 epo=0 ts=1608379270.906656340,1 min=1608379270.901269918,0 seq=9} lock=true stat=PENDING rts=1608379270.901269918,0 wto=false max=1608379271.401269918,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE TYPE public.enum48 AS ENUM ('ad', 'adad', 'fd', 'dadf', 's');
          |   SELECT 'validating all objects';
          | COMMIT; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=7a80f75d key=/Table/SystemConfigSpan/Start pri=0.05112607 epo=0 ts=1608379270.906656340,1 min=1608379270.901057171,0 seq=9} lock=true stat=PENDING rts=1608379270.901057171,0 wto=false max=1608379271.401057171,0 (SQLSTATE 40001)
          | BEGIN
          |   SELECT 'validating all objects';
          |   CREATE TYPE public.enum36 AS ENUM ('daaa', 'dds', 'dsas');
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2530481&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(jayshrivastava) tracked by https://github.com/cockroachdb/cockroach/issues/58068

(roachtest).schemachange/random-load failed on master@75efa1fe6f2096adc9db474026a2b7235e53c388:

          | (1) COMMAND_PROBLEM
          | Wraps: (2) Node 1. Command with error:
          |   | ```
          |   | ./workload run schemachange --verbose=1 --tolerate-errors=false  --histograms=perf/stats.json --max-ops 5000 --concurrency 20
          |   | ```
          | Wraps: (3) exit status 1
          | Error types: (1) errors.Cmd (2) *hintdetail.withDetail (3) *exec.ExitError
          |
          | stdout:
          | <... some data truncated by circular buffer; go to artifacts for details ...>
          | ema161" CASCADE;
          |   CREATE TABLE IF NOT EXISTS schema185.table265 (col265_266 INT4 NULL, col265_267 BIT(11) NOT NULL, col265_268 REGCLASS NULL, col265_269 REGPROC NOT NULL, col265_270 REGNAMESPACE NOT NULL, col265_271 TIMESTAMP NOT NULL, col265_272 REGPROC NOT NULL, col265_273 DECIMAL NOT NULL, col265_274 NAME NULL, col265_275 TIMETZ, col265_276 REGTYPE NOT NULL, PRIMARY KEY (col265_271 DESC, col265_276 DESC, col265_270 ASC, col265_267, col265_272 ASC, col265_273), UNIQUE (col265_275 ASC, col265_267 DESC) STORING (col265_266), INDEX (col265_274 DESC, col265_270, col265_273, col265_266, col265_272 DESC, col265_276 DESC, col265_271 DESC, col265_269 DESC, col265_267 ASC), UNIQUE (col265_276) STORING (col265_266, col265_269), INDEX (col265_273 DESC, col265_266 DESC, col265_271 DESC, col265_268 DESC, col265_272 DESC, col265_270, col265_274 ASC, col265_275, col265_267 ASC), UNIQUE (col265_272 DESC, col265_269 ASC), UNIQUE (col265_273 ASC, col265_269, col265_268 ASC, col265_275 DESC, col265_274 DESC) STORING (col265_266), INDEX (col265_269 DESC, col265_272 DESC, col265_268, col265_271 ASC, col265_270 DESC, col265_266 ASC, col265_276 DESC, col265_275 ASC), UNIQUE (col265_275, col265_271 ASC, col265_269 DESC, col265_270 DESC, col265_268 DESC), INDEX (col265_266 ASC, col265_271 DESC, col265_274 DESC, col265_276 ASC, col265_270, col265_268 ASC) STORING (col265_269, col265_275));
          |   SELECT 'validating all objects';
          |   CREATE SEQUENCE IF NOT EXISTS public.seq294;
          | COMMIT; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=227bb528 key=/Table/SystemConfigSpan/Start pri=0.05982053 epo=0 ts=1608465223.332781170,1 min=1608465223.316617532,0 seq=32} lock=true stat=PENDING rts=1608465223.316617532,0 wto=false max=1608465223.816617532,0 (SQLSTATE 40001)
          | BEGIN
          |   DROP SEQUENCE public.seq160;
          |   DROP SCHEMA "schema161" CASCADE;
          |   ALTER TABLE schema185.table295 ADD COLUMN IrrelevantColumnName string;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "schema185.table295" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   SELECT 'validating all objects';
          |   DROP SCHEMA "public" CASCADE;
          | ROLLBACK; expected SQLSTATE(S) 3F000, and got ERROR: cannot drop schema "public" (SQLSTATE 3F000)
          | BEGIN
          |   ALTER TABLE schema297.table298 ALTER COLUMN IrrelevantColumnName SET NOT NULL;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "schema297.table298" does not exist (SQLSTATE 42P01)
          | BEGIN
          |   CREATE SCHEMA schema185 AUTHORIZATION root;
          | ROLLBACK; expected SQLSTATE(S) 42P06, and got ERROR: schema "schema185" already exists (SQLSTATE 42P06)
          |     2.0s        0           45.0           51.0     12.1    159.4    335.5    335.5 opOk
          |     2.0s        0            7.0            9.5    335.5    604.0    604.0    604.0 txnCmtErr
          |     2.0s        0            1.9            2.0    469.8   1811.9   1811.9   1811.9 txnOk
          |     2.0s        0           32.0           28.5      3.7     75.5    201.3    201.3 txnRbk
          | BEGIN
          |   DROP SEQUENCE IF EXISTS public.seq160;
          |   CREATE SCHEMA IF NOT EXISTS schema22 AUTHORIZATION root;
          |   SELECT 'validating all objects';
          |   CREATE TABLE schema22.table300 (col300_301 TIMETZ NOT NULL, PRIMARY KEY (col300_301), UNIQUE (col300_301 DESC), UNIQUE (col300_301 DESC), UNIQUE (col300_301), INDEX (col300_301 DESC));
          | COMMIT; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): "sql txn" meta={id=3d67d201 key=/Table/SystemConfigSpan/Start pri=0.03188248 epo=0 ts=1608465223.454221372,1 min=1608465223.221454200,0 seq=18} lock=true stat=PENDING rts=1608465223.221454200,0 wto=false max=1608465223.721454200,0 (SQLSTATE 40001)
          | BEGIN
          |   ALTER SEQUENCE public.seq26 RENAME TO public.seq160;
          | COMMIT; 
          | BEGIN
          |   ALTER SEQUENCE public.seq160 RENAME TO public.seq278;
          |   ALTER TABLE schema22.table302 ALTER COLUMN IrrelevantColumnName DROP STORED;
          | ROLLBACK; expected SQLSTATE(S) 42P01, and got ERROR: relation "schema22.table302" does not exist (SQLSTATE 42P01)
          | BEGIN
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2531670&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(roachtest).schemachange/random-load failed on master@cd62a02833eeb02d14944668d09ee54806927a2b:

          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=6e71afdb key=/Table/SystemConfigSpan/Start pri=0.02904548 epo=0 ts=1608724998.765938652,1 min=1608724998.286795631,0 seq=19} lock=true stat=PENDING rts=1608724998.286795631,0 wto=false max=1608724998.786795631,0 (SQLSTATE 40001)",
          |  "txStatus": "TxStatusCommitFailure"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:03:18.255247",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table290 RENAME TO public.table436",
          |   "SELECT 'validating all objects'",
          |   "ALTER TABLE public.table436 DROP COLUMN \"col290_301\"",
          |   "CREATE INDEX index436_469 ON public.table436 (col290_303 ASC, col290_294 ASC, col290_306 DESC, col290_293 ASC, col290_302 ASC, col290_291, col290_297 DESC, col290_298, col290_304 DESC, col290_292 DESC, col290_299, col290_307 DESC, col290_295, col290_300 DESC)",
          |   "CREATE SCHEMA IF NOT EXISTS schema470 AUTHORIZATION root"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_PUSHER_ABORTED): \"sql txn\" meta={id=0ff3f8bf key=/Table/SystemConfigSpan/Start pri=0.01409356 epo=0 ts=1608724998.765938652,1 min=1608724998.255182356,0 seq=43} lock=true stat=ABORTED rts=1608724998.255182356,0 wto=false max=1608724998.255182356,0 (SQLSTATE 40001)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:03:18.145232",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table290 ADD CONSTRAINT table290_col290_291_unique UNIQUE (col290_291)",
          |   "CREATE TYPE public.enum471 AS ENUM ('af')",
          |   "CREATE TABLE schema472.table473 AS SELECT public.table290.col290_304, public.table290.col290_300, public.table290.col290_294, public.table290.col290_296, public.table290.col290_299, public.table290.col290_295, public.table290.col290_305, public.table290.col290_303, public.table290.col290_298, public.table290.col290_306, public.table290.col290_297, public.table290.col290_298, public.table290.col290_295, public.table290.col290_292, public.table290.col290_301, public.table290.col290_293, public.table290.col290_296, public.table290.col290_305, public.table290.col290_303, public.table290.col290_302, public.table290.col290_300, public.table290.col290_299, public.table290.col290_304, public.table290.col290_298, public.table290.col290_305, public.table290.col290_300 FROM public.table290, public.table290, public.table290"
          |  ],
          |  "expectedExecErrors": "3F000,42701,42712",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot create \"schema472.table473\" because the target database or schema does not exist (SQLSTATE 3F000)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:03:18.93998",
          |  "ops": [
          |   "BEGIN",
          |   "DROP SCHEMA \"schema435\" CASCADE"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: cannot rollback to savepoint after a transaction restart (SQLSTATE 40001)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2538659&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(roachtest).schemachange/random-load failed on master@152c3b30ae1dcb7a7cefc6aa7fa86c2dd264bd2e:

          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=9cccbf8b key=/Table/SystemConfigSpan/Start pri=0.03864379 epo=0 ts=1608896289.455806735,1 min=1608896289.049831921,0 seq=15} lock=true stat=PENDING rts=1608896289.049831921,0 wto=false max=1608896289.549831921,0 (SQLSTATE 40001)",
          |  "txStatus": "TxStatusCommitFailure"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:38:09.455875",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects'",
          |   "CREATE TABLE schema3.table141 (col141_142 REGPROCEDURE NULL, col141_143 FLOAT4 NOT NULL, col141_144 REGPROC NOT NULL, col141_145 BYTES NOT NULL, col141_146 OID NOT NULL, col141_147 TIMESTAMP NOT NULL, col141_148 FLOAT4 NULL, col141_149 FLOAT8 NOT NULL, col141_150 BIT(8) NOT NULL, col141_151 INET NOT NULL, col141_152 REGPROC NOT NULL, PRIMARY KEY (col141_144, col141_145 DESC, col141_146 ASC, col141_151 ASC, col141_149 DESC, col141_143), FAMILY (col141_150), FAMILY (col141_147, col141_152, col141_144), FAMILY (col141_145), FAMILY (col141_142, col141_143), FAMILY (col141_149, col141_146), FAMILY (col141_151), FAMILY (col141_148))",
          |   "CREATE TYPE public.enum155 AS ENUM ('add', 'sdaff', 'fdd', 'sad')",
          |   "ALTER TABLE schema3.table141 DROP CONSTRAINT \"primary\"",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "0A000",
          |  "message": "COMMIT; Successfully got expected commit error",
          |  "txStatus": "TxStatusCommitFailure"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:38:08.971468",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects'",
          |   "SELECT 'validating all objects'",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "",
          |  "txStatus": "TxStatusCommitSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:38:09.15234",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE schema61.table132 (col132_133 CHAR NOT NULL, col132_134 REGPROC NOT NULL, PRIMARY KEY (col132_134, col132_133), INDEX (col132_134), INDEX (col132_133), INDEX (col132_134 DESC), UNIQUE (col132_133 ASC), FAMILY (col132_133), FAMILY (col132_134))",
          |   "CREATE SCHEMA IF NOT EXISTS schema61 AUTHORIZATION root",
          |   "INSERT INTO schema61.table132 (col132_133,col132_134) VALUES ('n':::STRING,2031338374:::OID),(e'\\x07':::STRING,863487264:::OID),('E':::STRING,2896185826:::OID)",
          |   "ALTER TABLE schema61.table132 ADD CONSTRAINT table132_col132_133_table132_col132_133_fk FOREIGN KEY (col132_133) REFERENCES schema61.table132 (col132_133) ON DELETE CASCADE ON UPDATE CASCADE"
          |  ],
          |  "expectedExecErrors": "0A000",
          |  "expectedCommitErrors": "",
          |  "message": "***FAIL; Failed to receive an execution error when errors were expected",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2543440&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(roachtest).schemachange/random-load failed on master@4e3a9ccdb9cac89ae32d6248bb9b119b5bc250f7:

          |  "workerId": 0,
          |  "clientTimestamp": "11:48:25.818385",
          |  "ops": [
          |   "BEGIN",
          |   "INSERT INTO schema18.table296 (col180_181,col180_182,col180_183,col180_184,col180_185,col180_186) VALUES ('ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff':::INET,B'110101',e'\\x02}\\x06g2':::STRING:::NAME,'03:50:55.825535':::TIME,'1970-10-12 12:31:22.000492+00:00':::TIMESTAMPTZ,NULL),('6f24:6e15:7af0:fb64:c5a4:51a4:3ff6:895f/86':::INET,B'011111',e'Z\u003c\\x0e(`\\x07Tx_':::STRING:::NAME,'16:46:57.911515':::TIME,'2002-10-04 08:18:18.000334+00:00':::TIMESTAMPTZ,'BOX(-0.29156485106672375 0.0613202857507153,1.8449020212094904 1.6514957101109269)':::BOX2D),('91d3:f046:ce68:b433:3d15:d573:155b:cdd0/36':::INET,B'101110',e'u#}.\\x02\\x02':::STRING:::NAME,'00:00:00':::TIME,'1998-03-11 12:46:09.000715+00:00':::TIMESTAMPTZ,'BOX(-0.3128585745160769 -0.07909786003229935,0.5411337173827768 0.3564938265514457)':::BOX2D)",
          |   "CREATE TABLE schema18.table417 (col417_418 BIT(11) NOT NULL, col417_419 VARCHAR NOT NULL, col417_420 INT8, col417_421 TIME NULL, col417_422 OID, col417_423 FLOAT4 NOT NULL, col417_424 INTERVAL NOT NULL, col417_425 REGNAMESPACE NULL, PRIMARY KEY (col417_419, col417_418 DESC, col417_424, col417_423 ASC), UNIQUE (col417_421 ASC, col417_418, col417_425 ASC, col417_424), UNIQUE (col417_423, col417_419, col417_420 ASC, col417_418) STORING (col417_421, col417_422), UNIQUE (col417_419 ASC, col417_421 DESC, col417_425 ASC, col417_423, col417_424, col417_420 ASC, col417_422 DESC, col417_418), INDEX (col417_422 ASC, col417_420) STORING (col417_425), INDEX (col417_423 DESC, col417_425 ASC, col417_420, col417_422, col417_418 DESC, col417_419 DESC, col417_421), INDEX (col417_424 DESC, col417_418, col417_425 DESC, col417_423 DESC), INDEX (col417_420 DESC, col417_424 ASC, col417_425 DESC, col417_421 DESC, col417_423 ASC, col417_419 ASC), UNIQUE (col417_421 ASC))"
          |  ],
          |  "expectedExecErrors": "0A000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: unimplemented: the first schema change statement in a transaction must precede any writes (SQLSTATE 0A000)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:48:25.89917",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE INDEX index296_411 ON schema18.table296 (col180_186 DESC) STORING (col180_181, col180_184, col180_183, col180_182, col180_185)"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_WRITE_TOO_OLD - WriteTooOld flag converted to WriteTooOldError): \"sql txn\" meta={id=418582ed key=/Table/SystemConfigSpan/Start pri=0.00087451 epo=0 ts=1609069706.528263936,1 min=1609069705.899124434,0 seq=6} lock=true stat=PENDING rts=1609069705.899124434,0 wto=false max=1609069705.899124434,0 (SQLSTATE 40001)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:48:26.342157",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE UNIQUE INVERTED INDEX IF NOT EXISTS index296_416 ON schema18.table296 (col180_181 ASC, col180_184 ASC, col180_182 DESC, col180_185 DESC, col180_186) STORING (col180_183)"
          |  ],
          |  "expectedExecErrors": "0A000,26000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: inverted indexes don't support stored columns (SQLSTATE 26000)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:48:26.444025",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE schema18.table296 ADD CONSTRAINT table296_col180_181_table296_col180_181_fk FOREIGN KEY (col180_181) REFERENCES schema18.table296 (col180_181) ON DELETE CASCADE ON UPDATE CASCADE"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: there is no unique constraint matching given keys for referenced table table296 (SQLSTATE 23503)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2544831&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(roachtest).schemachange/random-load failed on master@08c89a597a06520c30faf01965f9c74fe9b9854f:

          |  "ops": [
          |   "BEGIN",
          |   "INSERT INTO schema261.table291 (col291_292,col291_293,col291_294,col291_295) VALUES ('2029-06-17 15:35:13.000763':::TIMESTAMP,'8901122e-f927-4cb0-b5d2-f01fb2f9d715':::UUID,(-7.130562951563524570E+20):::DECIMAL,'294276-12-31 23:59:59.999999':::TIMESTAMP)",
          |   "CREATE TABLE schema261.table309 (col309_310 NAME NOT NULL, col309_311 TIMETZ NOT NULL, col309_312 BYTES NULL, col309_313 VARCHAR NULL, col309_314 TIMETZ, col309_315 FLOAT4 NOT NULL, col309_316 REGNAMESPACE NOT NULL, col309_317 REGPROC NOT NULL, col309_318 STRING NULL, col309_319 BOX2D NOT NULL, col309_320 VARCHAR NOT NULL, col309_321 STRING, col309_322 BOX2D NOT NULL, col309_323 BOX2D NOT NULL, col309_324 UUID NOT NULL, col309_325 REGPROC NOT NULL, PRIMARY KEY (col309_310 DESC, col309_325 ASC, col309_323, col309_317 ASC, col309_316 DESC, col309_311 DESC, col309_324 ASC, col309_320 DESC, col309_315 DESC, col309_319 ASC), UNIQUE (col309_312 DESC, col309_322, col309_323 ASC, col309_319, col309_311 DESC, col309_313 DESC, col309_317, col309_318 ASC, col309_321 DESC), INDEX (col309_321 DESC, col309_313 DESC, col309_320 ASC, col309_315 ASC, col309_312 DESC, col309_310 DESC, col309_316 DESC))"
          |  ],
          |  "expectedExecErrors": "0A000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: unimplemented: the first schema change statement in a transaction must precede any writes (SQLSTATE 0A000)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:56:27.113879",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects'",
          |   "SELECT 'validating all objects'",
          |   "CREATE TABLE schema286.table326 AS SELECT schema261.table291.col291_292, \"IrrelevantColumnName\", schema261.table291.col291_294, schema261.table291.col291_292 FROM schema261.table291, public.table308, schema261.table291"
          |  ],
          |  "expectedExecErrors": "42701,42712,42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"public.table308\" does not exist (SQLSTATE 42P01)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:56:27.299987",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE schema261.table291 ALTER COLUMN \"col291_295\" DROP STORED"
          |  ],
          |  "expectedExecErrors": "42611",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: column \"col291_295\" is not a computed column (SQLSTATE 42611)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:56:27.235767",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA schema302 AUTHORIZATION root"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: cannot rollback to savepoint after a transaction restart (SQLSTATE 40001)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2546130&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(roachtest).schemachange/random-load failed on master@08c89a597a06520c30faf01965f9c74fe9b9854f:

          |  "workerId": 0,
          |  "clientTimestamp": "17:55:32.692042",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table24 DROP COLUMN \"col24_25\""
          |  ],
          |  "expectedExecErrors": "42P10",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: column \"col24_25\" is referenced by the primary key (SQLSTATE 42P10)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "17:55:32.634621",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table24 DROP COLUMN \"col24_299\""
          |  ],
          |  "expectedExecErrors": "42703",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: column \"col24_299\" does not exist (SQLSTATE 42703)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "17:55:32.512482",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER TABLE public.table24 DROP CONSTRAINT \"table24_col24_26_col24_28_key\""
          |  ],
          |  "expectedExecErrors": "0A000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: unimplemented: cannot drop UNIQUE constraint \"table24_col24_26_col24_28_key\" using ALTER TABLE DROP CONSTRAINT, use DROP INDEX CASCADE instead (SQLSTATE 0A000)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "17:55:32.512911",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE IF NOT EXISTS public.table286 (col286_287 INT8[] NULL, col286_288 \"char\" NOT NULL, col286_289 INT4, col286_290 OID NOT NULL, col286_291 BIT(43) NULL, col286_292 BIT(11) NOT NULL, col286_293 OID[] NULL, col286_294 TIME NOT NULL, col286_295 OID, col286_296 NAME, col286_297 INT2 NOT NULL, col286_298 INT8 NOT NULL, PRIMARY KEY (col286_292 ASC, col286_294 ASC, col286_290 ASC, col286_288 ASC, col286_298, col286_297), INDEX (col286_297 DESC, col286_296 ASC, col286_292 ASC, col286_288) STORING (col286_287), INDEX (col286_298 DESC, col286_294 DESC, col286_288 DESC, col286_296 DESC, col286_292 DESC, col286_295 ASC, col286_297, col286_289) STORING (col286_287), UNIQUE (col286_290) STORING (col286_287), INDEX (col286_292, col286_289, col286_298, col286_291 ASC))",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq131 OWNED BY schema175.table306.\"IrrelevantColumnName\""
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "***FAIL; Failed to receive an execution error when errors were expected",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2547217&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(roachtest).schemachange/random-load failed on master@08c89a597a06520c30faf01965f9c74fe9b9854f:

          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA schema43 AUTHORIZATION root",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "",
          |  "txStatus": "TxStatusCommitSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "01:09:12.450941",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects'",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "",
          |  "txStatus": "TxStatusCommitSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "01:09:12.327942",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SEQUENCE IF NOT EXISTS public.seq168",
          |   "ALTER TABLE schema5.table199 DROP COLUMN \"IrrelevantColumnName\""
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema5.table199\" does not exist (SQLSTATE 42P01)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "01:09:12.513257",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE schema43.table195 AS SELECT \"IrrelevantColumnName\" FROM schema5.table173"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Received an unexpected execution error: ERROR: descriptor is being dropped (SQLSTATE XXUUU)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2547724&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(roachtest).schemachange/random-load failed on master@365c5504b75c9a9260365a628a5110c48312178b:

          |  "ops": [
          |   "BEGIN",
          |   "CREATE TYPE schema60.enum61 AS ENUM ('fadsa')"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot create \"schema60.enum61\" because the target database or schema does not exist (SQLSTATE 3F000)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:08:20.664599",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects'",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "",
          |  "txStatus": "TxStatusCommitSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:08:20.665307",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects'",
          |   "CREATE TABLE public.table42 (col42_43 BIT(27) NOT NULL, col42_44 OID NOT NULL, col42_45 REGCLASS NOT NULL, col42_46 REGPROCEDURE NULL, col42_47 INT2 NULL, col42_48 REGPROCEDURE NULL, col42_49 DECIMAL, col42_50 BYTES, col42_51 BIT(4), col42_52 REGTYPE NOT NULL, col42_53 REGPROCEDURE NOT NULL, UNIQUE (col42_45 ASC, col42_44 DESC) STORING (col42_46, col42_51, col42_52), UNIQUE (col42_50 ASC, col42_44 DESC, col42_49 ASC, col42_51 ASC, col42_43 ASC, col42_46 DESC, col42_48 ASC, col42_52 DESC, col42_47 ASC, col42_53), UNIQUE (col42_47 ASC, col42_50, col42_46 DESC) STORING (col42_43, col42_44, col42_45, col42_51, col42_53), INDEX (col42_43, col42_45 DESC, col42_49 DESC, col42_53, col42_50, col42_51 ASC, col42_44, col42_52 ASC), INDEX (col42_50 DESC, col42_48, col42_46 DESC, col42_49, col42_43, col42_47 ASC, col42_45), UNIQUE (col42_51, col42_48 ASC, col42_45 ASC) STORING (col42_43, col42_46, col42_52), UNIQUE (col42_51 ASC, col42_46, col42_53, col42_43 ASC, col42_44 DESC, col42_50, col42_49 DESC), INDEX (col42_43 DESC), UNIQUE (col42_45 ASC), FAMILY (col42_45, col42_43), FAMILY (col42_51), FAMILY (col42_44, col42_46), FAMILY (col42_48, col42_53), FAMILY (col42_49, col42_52), FAMILY (col42_50, col42_47))",
          |   "ALTER TABLE public.table42 DROP COLUMN \"col42_59\""
          |  ],
          |  "expectedExecErrors": "42703",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: column \"col42_59\" does not exist (SQLSTATE 42703)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "12:08:20.965128",
          |  "ops": [
          |   "BEGIN",
          |   "DROP SCHEMA \"schema23\" CASCADE"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: cannot rollback to savepoint after a transaction restart (SQLSTATE 40001)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2548186&tab=artifacts#/schemachange/random-load) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(roachtest).schemachange/random-load failed on master@6d49a323b52966becfe8a2c38a1a8ccdf8ee58a1:

          |   "ALTER TABLE public.table324 ADD COLUMN col324_434 CHAR NOT NULL",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "TXN RETRY ERROR; ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_SERIALIZABLE - failed preemptive refresh): \"sql txn\" meta={id=27d5d2fe key=/Table/SystemConfigSpan/Start pri=0.00821259 epo=0 ts=1609328929.926927939,1 min=1609328929.514401783,0 seq=18} lock=true stat=PENDING rts=1609328929.514401783,0 wto=false max=1609328930.014401783,0 (SQLSTATE 40001)",
          |  "txStatus": "TxStatusCommitFailure"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:48:49.767743",
          |  "ops": [
          |   "BEGIN",
          |   "ALTER SEQUENCE public.seq318 RENAME TO public.seq318",
          |   "SELECT 'validating all objects'",
          |   "ALTER SEQUENCE public.seq318 RENAME TO public.seq318",
          |   "DROP SCHEMA \"public\" CASCADE"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot drop schema \"public\" (SQLSTATE 3F000)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:48:49.616656",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE TABLE public.table403 (col403_404 FLOAT8 NOT NULL, col403_405 UUID NULL, col403_406 TIMESTAMPTZ, col403_407 INT4 NOT NULL, col403_408 TIMESTAMPTZ NULL, PRIMARY KEY (col403_404 ASC), INDEX (col403_407, col403_406 ASC) STORING (col403_405, col403_408), UNIQUE (col403_406, col403_407 DESC, col403_405 ASC, col403_408 DESC, col403_404 DESC), UNIQUE (col403_408 ASC, col403_407, col403_404 ASC, col403_406 ASC, col403_405 DESC), UNIQUE (col403_405 DESC), FAMILY (col403_404), FAMILY (col403_408, col403_405), FAMILY (col403_407, col403_406))",
          |   "ALTER TABLE public.table403 ALTER COLUMN \"col403_407\" DROP STORED"
          |  ],
          |  "expectedExecErrors": "42611",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: column \"col403_407\" is not a computed column (SQLSTATE 42611)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:48:49.943569",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA IF NOT EXISTS schema392 AUTHORIZATION root"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: cannot rollback to savepoint after a transaction restart (SQLSTATE 40001)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2550095&tab=artifacts#/schemachange/random-load) Related: - #58344 roachtest: schemachange/random-load failed [C-test-failure](https://api.github.com/repos/cockroachdb/cockroach/labels/C-test-failure) [O-roachtest](https://api.github.com/repos/cockroachdb/cockroach/labels/O-roachtest) [O-robot](https://api.github.com/repos/cockroachdb/cockroach/labels/O-robot) [branch-release-20.2](https://api.github.com/repos/cockroachdb/cockroach/labels/branch-release-20.2) [release-blocker](https://api.github.com/repos/cockroachdb/cockroach/labels/release-blocker) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)

cockroach-teamcity commented 3 years ago

(roachtest).schemachange/random-load failed on master@b93fd531b93cb010729cb73fe679cdff9388cf27:

          |   "BEGIN",
          |   "CREATE SCHEMA schema146 AUTHORIZATION root",
          |   "SELECT 'validating all objects'",
          |   "DROP SCHEMA \"public\" CASCADE"
          |  ],
          |  "expectedExecErrors": "3F000",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: cannot drop schema \"public\" (SQLSTATE 3F000)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:30:11.277232",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA IF NOT EXISTS schema4 AUTHORIZATION root",
          |   "COMMIT"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "",
          |  "txStatus": "TxStatusCommitSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:30:11.932018",
          |  "ops": [
          |   "BEGIN",
          |   "SELECT 'validating all objects'",
          |   "ALTER TABLE schema4.table195 ADD CONSTRAINT IrrelevantConstraintName UNIQUE (IrrelevantColumnName)"
          |  ],
          |  "expectedExecErrors": "42P01",
          |  "expectedCommitErrors": "",
          |  "message": "ROLLBACK; Successfully got expected execution error: ERROR: relation \"schema4.table195\" does not exist (SQLSTATE 42P01)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
          | {
          |  "workerId": 0,
          |  "clientTimestamp": "11:30:12.225419",
          |  "ops": [
          |   "BEGIN",
          |   "CREATE SCHEMA schema172 AUTHORIZATION root"
          |  ],
          |  "expectedExecErrors": "",
          |  "expectedCommitErrors": "",
          |  "message": "***UNEXPECTED ERROR; Failed to generate a random operation: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: cannot rollback to savepoint after a transaction restart (SQLSTATE 40001)",
          |  "txStatus": "TxStatusRollbackSuccess"
          | }
        Wraps: (4) exit status 20
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *main.withCommandDetails (4) *exec.ExitError
More

Artifacts: [/schemachange/random-load](https://teamcity.cockroachdb.com/viewLog.html?buildId=2552340&tab=artifacts#/schemachange/random-load) Related: - #58344 roachtest: schemachange/random-load failed [C-test-failure](https://api.github.com/repos/cockroachdb/cockroach/labels/C-test-failure) [O-roachtest](https://api.github.com/repos/cockroachdb/cockroach/labels/O-roachtest) [O-robot](https://api.github.com/repos/cockroachdb/cockroach/labels/O-robot) [branch-release-20.2](https://api.github.com/repos/cockroachdb/cockroach/labels/branch-release-20.2) [release-blocker](https://api.github.com/repos/cockroachdb/cockroach/labels/release-blocker) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2Aschemachange%2Frandom-load.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)