search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.88k stars 3.77k forks source link

pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [duplicate key value violates unique constraint] #129909

Closed github-actions[bot] closed 5 days ago

github-actions[bot] commented 2 weeks ago

pkg/ccl/testccl/workload/schemachange/schemachange_test.TestWorkload failed on master @ 4756b021edfdd477427f4ec34bf83f837f26a855:

   {
    "query": "SELECT EXISTS (SELECT * FROM [SHOW REGIONS FROM DATABASE])",
    "result": false
   },
   {
    "query": "WITH descriptors AS (SELECT crdb_internal.pb_to_json('cockroach.sql.sqlbase.Descriptor', descriptor)-\u003e'table' AS d FROM system.descriptor WHERE id = $1::REGCLASS) SELECT EXISTS (SELECT 1 FROM descriptors WHERE ((d-\u003e'localityConfig')-\u003e'regionalByRow') IS NOT NULL)",
    "queryArgs": [
     "schema_w0_23.table_w1_70"
    ],
    "result": false
   }
  ],
  "previousStatements": [
   "ALTER TABLE schema_w0_23.table_w1_70 ADD CONSTRAINT \"table_w1_70_col70_w1_80_unique\" UNIQUE (col70_w1_80)"
  ]
 }
}
    schema_change_external_test.go:134: 
            Error Trace:    pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:134
            Error:          Received unexpected error:
                            ***UNEXPECTED COMMIT ERROR; Received an unexpected commit error: ERROR: transaction committed but schema change aborted with error: (23505): duplicate key value violates unique constraint "table_w1_70_col70_w1_80_unique" (SQLSTATE 23505)
                            (1) forced error mark
                              | "fatal error when running txn"
                              | github.com/cockroachdb/errors/withstack/*withstack.withStack::
                            Wraps: (2)
                            Wraps: (3) attached stack trace
                              -- stack trace:
                              | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*schemaChangeWorker).run
                              |     github.com/cockroachdb/cockroach/pkg/workload/schemachange/schemachange.go:658
                              | pkg/ccl/testccl/workload/schemachange/schemachange_test_test.TestWorkload.TestWorkload.func4.func6
                              |     pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:123
                              | golang.org/x/sync/errgroup.(*Group).Go.func1
                              |     golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78
                              | runtime.goexit
                              |     src/runtime/asm_amd64.s:1695
                            Wraps: (4) ***UNEXPECTED COMMIT ERROR; Received an unexpected commit error
                            Wraps: (5) ERROR: transaction committed but schema change aborted with error: (23505): duplicate key value violates unique constraint "table_w1_70_col70_w1_80_unique" (SQLSTATE 23505)
                            Error types: (1) *markers.withMark (2) *schemachange.ErrorState (3) *withstack.withStack (4) *errutil.withPrefix (5) *pgconn.PgError
            Test:           TestWorkload
Schema Workload Stats
Total Schema Statements Executed = 99
Total Schema Statements Succeeded = 99
Total Schema Statement Expected Failures = 0
Total Transactions Committed = 72
Total Transactions Rolled Back = 51
Total Transactions Executed = 123
    schema_change_external_test.go:99: backup, tracing data, and system table dumps in /var/lib/engflow/worker/work/2/exec/bazel-out/k8-fastbuild/testlogs/pkg/ccl/testccl/workload/schemachange/schemachange_test/test.outputs/logTestWorkload4057762091
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestWorkload4057762091
--- FAIL: TestWorkload (22.10s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/sql-foundations

This test on roachdash | Improve this report!

Jira issue: CRDB-41764

cockroach-teamcity commented 2 weeks ago

pkg/ccl/testccl/workload/schemachange/schemachange_test.TestWorkload failed on master @ 5b1702fc72dba825e94c60ac581f7f2bdc04d1fb:

   {
    "query": "SELECT EXISTS (SELECT * FROM [SHOW REGIONS FROM DATABASE])",
    "result": false
   },
   {
    "query": "WITH descriptors AS (SELECT crdb_internal.pb_to_json('cockroach.sql.sqlbase.Descriptor', descriptor)-\u003e'table' AS d FROM system.descriptor WHERE id = $1::REGCLASS) SELECT EXISTS (SELECT 1 FROM descriptors WHERE ((d-\u003e'localityConfig')-\u003e'regionalByRow') IS NOT NULL)",
    "queryArgs": [
     "public.table_w0_76"
    ],
    "result": false
   }
  ],
  "previousStatements": [
   "ALTER TABLE public.table_w0_76 ADD CONSTRAINT \"table_w0_76_col26_w1_33_unique\" UNIQUE (col26_w1_33)"
  ]
 }
}
    schema_change_external_test.go:134: 
            Error Trace:    pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:134
            Error:          Received unexpected error:
                            ***UNEXPECTED COMMIT ERROR; Received an unexpected commit error: ERROR: transaction committed but schema change aborted with error: (23505): duplicate key value violates unique constraint "table_w0_76_col26_w1_33_unique" (SQLSTATE 23505)
                            (1) forced error mark
                              | "fatal error when running txn"
                              | github.com/cockroachdb/errors/withstack/*withstack.withStack::
                            Wraps: (2)
                            Wraps: (3) attached stack trace
                              -- stack trace:
                              | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*schemaChangeWorker).run
                              |     github.com/cockroachdb/cockroach/pkg/workload/schemachange/schemachange.go:658
                              | pkg/ccl/testccl/workload/schemachange/schemachange_test_test.TestWorkload.TestWorkload.func4.func6
                              |     pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:123
                              | golang.org/x/sync/errgroup.(*Group).Go.func1
                              |     golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78
                              | runtime.goexit
                              |     src/runtime/asm_amd64.s:1695
                            Wraps: (4) ***UNEXPECTED COMMIT ERROR; Received an unexpected commit error
                            Wraps: (5) ERROR: transaction committed but schema change aborted with error: (23505): duplicate key value violates unique constraint "table_w0_76_col26_w1_33_unique" (SQLSTATE 23505)
                            Error types: (1) *markers.withMark (2) *schemachange.ErrorState (3) *withstack.withStack (4) *errutil.withPrefix (5) *pgconn.PgError
            Test:           TestWorkload
Schema Workload Stats
Total Schema Statements Executed = 52
Total Schema Statements Succeeded = 52
Total Schema Statement Expected Failures = 0
Total Transactions Committed = 32
Total Transactions Rolled Back = 28
Total Transactions Executed = 60
    schema_change_external_test.go:99: backup, tracing data, and system table dumps in /var/lib/engflow/worker/work/3/exec/bazel-out/k8-fastbuild/testlogs/pkg/ccl/testccl/workload/schemachange/schemachange_test/run_15_of_25/test.outputs/logTestWorkload1831917458
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestWorkload1831917458
--- FAIL: TestWorkload (13.59s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #129462 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot P-2 T-sql-foundations branch-release-24.2.1-rc] - #128953 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed ["enum type does not exist" error code when setting a DEFAULT] [C-test-failure O-robot P-3 T-sql-foundations branch-release-23.2.10-rc]

This test on roachdash | Improve this report!

rafiss commented 2 weeks ago

similar to https://github.com/cockroachdb/cockroach/issues/129366 cc @fqazi

cockroach-teamcity commented 2 weeks ago

pkg/ccl/testccl/workload/schemachange/schemachange_test.TestWorkload failed on master @ 8551145a0c99c4c95a28ec470e699d0c20ca97ab:

   {
    "query": "SELECT EXISTS (SELECT * FROM schema_w1_28.table_w0_64)",
    "result": false
   },
   {
    "query": "WITH descriptors AS (SELECT crdb_internal.pb_to_json('cockroach.sql.sqlbase.Descriptor', descriptor)-\u003e'table' AS d FROM system.descriptor WHERE id = $1::REGCLASS) SELECT EXISTS (SELECT mut FROM (SELECT json_array_elements(d-\u003e'mutations') AS mut FROM descriptors) WHERE (mut-\u003e'primaryKeySwap') IS NOT NULL)",
    "queryArgs": [
     "schema_w1_28.table_w0_64"
    ],
    "result": false
   }
  ],
  "previousStatements": [
   "ALTER TABLE schema_w1_28.table_w0_64 ADD COLUMN col_w0_64_w0_70 VARCHAR NOT NULL"
  ]
 }
}
    schema_change_external_test.go:134: 
            Error Trace:    pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:134
            Error:          Received unexpected error:
                            ***UNEXPECTED COMMIT ERROR; Received an unexpected commit error: ERROR: transaction committed but schema change aborted with error: (23502): validation of column "col_w0_64_w0_70" NOT NULL failed on row: ".co.l64_w0_65"='i', "col/64_w0_66"=0.28295178868727344, col6_4_w0_67=3802, col64_w0_68=3220, col_w0_64_w0_70=NULL (SQLSTATE 23502)
                            (1) forced error mark
                              | "fatal error when running txn"
                              | github.com/cockroachdb/errors/withstack/*withstack.withStack::
                            Wraps: (2)
                            Wraps: (3) attached stack trace
                              -- stack trace:
                              | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*schemaChangeWorker).run
                              |     github.com/cockroachdb/cockroach/pkg/workload/schemachange/schemachange.go:658
                              | pkg/ccl/testccl/workload/schemachange/schemachange_test_test.TestWorkload.TestWorkload.func4.func6
                              |     pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:123
                              | golang.org/x/sync/errgroup.(*Group).Go.func1
                              |     golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78
                              | runtime.goexit
                              |     src/runtime/asm_amd64.s:1695
                            Wraps: (4) ***UNEXPECTED COMMIT ERROR; Received an unexpected commit error
                            Wraps: (5) ERROR: transaction committed but schema change aborted with error: (23502): validation of column "col_w0_64_w0_70" NOT NULL failed on row: ".co.l64_w0_65"='i', "col/64_w0_66"=0.28295178868727344, col6_4_w0_67=3802, col64_w0_68=3220, col_w0_64_w0_70=NULL (SQLSTATE 23502)
                            Error types: (1) *markers.withMark (2) *schemachange.ErrorState (3) *withstack.withStack (4) *errutil.withPrefix (5) *pgconn.PgError
            Test:           TestWorkload
Schema Workload Stats
Total Schema Statements Executed = 40
Total Schema Statements Succeeded = 40
Total Schema Statement Expected Failures = 0
Total Transactions Committed = 24
Total Transactions Rolled Back = 24
Total Transactions Executed = 48
    schema_change_external_test.go:99: backup, tracing data, and system table dumps in /var/lib/engflow/worker/work/3/exec/bazel-out/k8-fastbuild/testlogs/pkg/ccl/testccl/workload/schemachange/schemachange_test/run_15_of_25/test.outputs/logTestWorkload2769255836
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestWorkload2769255836
--- FAIL: TestWorkload (10.49s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #129462 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot P-2 T-sql-foundations branch-release-24.2.1-rc] - #128953 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed ["enum type does not exist" error code when setting a DEFAULT] [C-test-failure O-robot P-3 T-sql-foundations branch-release-23.2.10-rc]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

pkg/ccl/testccl/workload/schemachange/schemachange_test.TestWorkload failed on master @ 4142920c2d5c50c0520c124764aeeda94ba043ae:

                              |   |     github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/store_send.go:193
                              |   | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Stores).SendWithWriteBytes
                              |   |     github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/stores.go:202
                              |   | github.com/cockroachdb/cockroach/pkg/server.(*Node).batchInternal
                              |   |     github.com/cockroachdb/cockroach/pkg/server/node.go:1543
                              |   | github.com/cockroachdb/cockroach/pkg/server.(*Node).Batch
                              |   |     github.com/cockroachdb/cockroach/pkg/server/node.go:1744
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.func1
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:708
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.ServerInterceptor.func12
                              |   |     github.com/cockroachdb/cockroach/pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:97
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:819
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func3
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:170
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:819
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.kvAuth.unaryInterceptor
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/auth.go:95
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:819
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1.1
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:137
                              |   | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr
                              |   |     github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:336
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:135
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:819
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.func2
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:718
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.NewContext.ClientInterceptor.func8
                              |   |     github.com/cockroachdb/cockroach/pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:228
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.getChainUnaryInvoker.func1
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:903
                              | Wraps: (2) single entry value with no default column id
                              | Error types: (1) *withstack.withStack (2) *errutil.leafError (SQLSTATE XXUUU)
                            Error types: (1) *markers.withMark (2) *schemachange.ErrorState (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.withPrefix (7) *pgconn.PgError
            Test:           TestWorkload
Schema Workload Stats
Total Schema Statements Executed = 125
Total Schema Statements Succeeded = 125
Total Schema Statement Expected Failures = 0
Total Transactions Committed = 82
Total Transactions Rolled Back = 75
Total Transactions Executed = 157
    schema_change_external_test.go:99: backup, tracing data, and system table dumps in /var/lib/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/testlogs/pkg/ccl/testccl/workload/schemachange/schemachange_test/run_19_of_25/test.outputs/logTestWorkload1968110049
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestWorkload1968110049
--- FAIL: TestWorkload (24.50s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #129462 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot P-2 T-sql-foundations branch-release-24.2.1-rc] - #128953 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed ["enum type does not exist" error code when setting a DEFAULT] [C-test-failure O-robot P-3 T-sql-foundations branch-release-23.2.10-rc]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

pkg/ccl/testccl/workload/schemachange/schemachange_test.TestWorkload failed on master @ f341fb5d738192dbd9ea55e3fd7ed9d52407f6f5:

   {
    "query": "WITH descriptors AS (SELECT crdb_internal.pb_to_json('cockroach.sql.sqlbase.Descriptor', descriptor)-\u003e'table' AS d FROM system.descriptor WHERE id = $1::REGCLASS) SELECT EXISTS (SELECT mut FROM (SELECT json_array_elements(d-\u003e'mutations') AS mut FROM descriptors) WHERE (mut-\u003e'primaryKeySwap') IS NOT NULL)",
    "queryArgs": [
     "public.table_w1_3"
    ],
    "result": false
   },
   {
    "query": "SELECT EXISTS (SELECT * FROM [SHOW REGIONS FROM DATABASE])",
    "result": false
   }
  ],
  "previousStatements": [
   "CREATE UNIQUE INDEX index_w1_3_w0_169 ON public.table_w1_3 (c_ol3_w1_6) STORING (\"col3 _w1_5\", col_w1_3_w0_8, col3_w1_4)"
  ]
 }
}
    schema_change_external_test.go:134: 
            Error Trace:    pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:134
            Error:          Received unexpected error:
                            ***UNEXPECTED COMMIT ERROR; Received an unexpected commit error: ERROR: transaction committed but schema change aborted with error: (23505): failed to ingest index entries during backfill: duplicate key value violates unique constraint "index_w1_3_w0_169" (SQLSTATE 23505)
                            (1) forced error mark
                              | "fatal error when running txn"
                              | github.com/cockroachdb/errors/withstack/*withstack.withStack::
                            Wraps: (2)
                            Wraps: (3) attached stack trace
                              -- stack trace:
                              | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*schemaChangeWorker).run
                              |     github.com/cockroachdb/cockroach/pkg/workload/schemachange/schemachange.go:658
                              | pkg/ccl/testccl/workload/schemachange/schemachange_test_test.TestWorkload.TestWorkload.func4.func6
                              |     pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:123
                              | golang.org/x/sync/errgroup.(*Group).Go.func1
                              |     golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78
                              | runtime.goexit
                              |     src/runtime/asm_amd64.s:1695
                            Wraps: (4) ***UNEXPECTED COMMIT ERROR; Received an unexpected commit error
                            Wraps: (5) ERROR: transaction committed but schema change aborted with error: (23505): failed to ingest index entries during backfill: duplicate key value violates unique constraint "index_w1_3_w0_169" (SQLSTATE 23505)
                            Error types: (1) *markers.withMark (2) *schemachange.ErrorState (3) *withstack.withStack (4) *errutil.withPrefix (5) *pgconn.PgError
            Test:           TestWorkload
Schema Workload Stats
Total Schema Statements Executed = 107
Total Schema Statements Succeeded = 107
Total Schema Statement Expected Failures = 1
Total Transactions Committed = 77
Total Transactions Rolled Back = 67
Total Transactions Executed = 144
    schema_change_external_test.go:99: backup, tracing data, and system table dumps in /var/lib/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/testlogs/pkg/ccl/testccl/workload/schemachange/schemachange_test/run_15_of_25/test.outputs/logTestWorkload4142447864
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestWorkload4142447864
--- FAIL: TestWorkload (22.75s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #129462 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot P-2 T-sql-foundations branch-release-24.2.1-rc] - #128953 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed ["enum type does not exist" error code when setting a DEFAULT] [C-test-failure O-robot P-3 T-sql-foundations branch-release-23.2.10-rc]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

pkg/ccl/testccl/workload/schemachange/schemachange_test.TestWorkload failed on master @ 950a090603bffa9216f01b03aeeb7dd093c6be64:

   {
    "query": "SELECT EXISTS (SELECT * FROM [SHOW REGIONS FROM DATABASE])",
    "result": false
   },
   {
    "query": "WITH descriptors AS (SELECT crdb_internal.pb_to_json('cockroach.sql.sqlbase.Descriptor', descriptor)-\u003e'table' AS d FROM system.descriptor WHERE id = $1::REGCLASS) SELECT EXISTS (SELECT 1 FROM descriptors WHERE ((d-\u003e'localityConfig')-\u003e'regionalByRow') IS NOT NULL)",
    "queryArgs": [
     "public.table_w1_51"
    ],
    "result": false
   }
  ],
  "previousStatements": [
   "ALTER TABLE public.table_w1_51 ADD CONSTRAINT \"table_w1_51_col59_w0_64_unique\" UNIQUE (col59_w0_64)"
  ]
 }
}
    schema_change_external_test.go:134: 
            Error Trace:    pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:134
            Error:          Received unexpected error:
                            ***UNEXPECTED COMMIT ERROR; Received an unexpected commit error: ERROR: transaction committed but schema change aborted with error: (23505): failed to ingest index entries during backfill: duplicate key value violates unique constraint "table_w1_51_col59_w0_64_unique" (SQLSTATE 23505)
                            (1) forced error mark
                              | "fatal error when running txn"
                              | github.com/cockroachdb/errors/withstack/*withstack.withStack::
                            Wraps: (2)
                            Wraps: (3) attached stack trace
                              -- stack trace:
                              | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*schemaChangeWorker).run
                              |     github.com/cockroachdb/cockroach/pkg/workload/schemachange/schemachange.go:658
                              | pkg/ccl/testccl/workload/schemachange/schemachange_test_test.TestWorkload.TestWorkload.func4.func6
                              |     pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:123
                              | golang.org/x/sync/errgroup.(*Group).Go.func1
                              |     golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78
                              | runtime.goexit
                              |     src/runtime/asm_amd64.s:1695
                            Wraps: (4) ***UNEXPECTED COMMIT ERROR; Received an unexpected commit error
                            Wraps: (5) ERROR: transaction committed but schema change aborted with error: (23505): failed to ingest index entries during backfill: duplicate key value violates unique constraint "table_w1_51_col59_w0_64_unique" (SQLSTATE 23505)
                            Error types: (1) *markers.withMark (2) *schemachange.ErrorState (3) *withstack.withStack (4) *errutil.withPrefix (5) *pgconn.PgError
            Test:           TestWorkload
Schema Workload Stats
Total Schema Statements Executed = 66
Total Schema Statements Succeeded = 66
Total Schema Statement Expected Failures = 0
Total Transactions Committed = 42
Total Transactions Rolled Back = 35
Total Transactions Executed = 77
    schema_change_external_test.go:99: backup, tracing data, and system table dumps in /var/lib/engflow/worker/work/3/exec/bazel-out/k8-fastbuild/testlogs/pkg/ccl/testccl/workload/schemachange/schemachange_test/run_1_of_25/test.outputs/logTestWorkload3110114137
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestWorkload3110114137
--- FAIL: TestWorkload (14.95s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #129462 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot P-2 T-sql-foundations branch-release-24.2.1-rc] - #128953 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed ["enum type does not exist" error code when setting a DEFAULT] [C-test-failure O-robot P-3 T-sql-foundations branch-release-23.2.10-rc]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

pkg/ccl/testccl/workload/schemachange/schemachange_test.TestWorkload failed on master @ fa9c0528fc0d06be1b4cfc534ec0501448111fbe:

    "query": "SELECT EXISTS (SELECT table_name FROM information_schema.views WHERE (table_schema = $1) AND (table_name = $2))",
    "queryArgs": [
     "public",
     "view_w1_126"
    ],
    "result": false
   }
  ],
  "previousStatements": [
   "CREATE VIEW public.view_w1_126 AS SELECT public.table_w0_111.crdb_internal_mvcc_timestamp, public.table_w1_86.\"c%v\fol86_w1_90\", public.table_w1_86.col86_w1_87, public.table_w1_86.\"col8\\\\xf06_w1_99\", public.table_w1_86.col86_w1_97, public.table_w1_86.col8😲6_w1_96, public.table_w1_86.col86_w1_102, public.table_w1_86.col86_w1_94, public.table_w1_86.\"%pcol86_w1_88\", public.table_w1_86.\"c|ol86_w1_89\", public.table_w1_86.\"col86\\v_w1_91\", public.table_w1_86.\"'col86_w1_93\", public.table_w1_86.cͪol86_w1_95, public.table_w1_86.col86_w1_98, public.table_w1_86.\"\\fcol8\\v6 _w1_100\", public.table_w1_86.col86_w1_92, public.table_w1_86.col86_w1_101 FROM public.table_w0_111, public.table_w1_86"
  ]
 }
}
    schema_change_external_test.go:134: 
            Error Trace:    pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:134
            Error:          Received unexpected error:
                            ***UNEXPECTED ERROR; Received an unexpected execution error.: ERROR: relation "view_w1_126" (138): column name "crdb_internal_mvcc_timestamp" conflicts with a system column name (SQLSTATE 42701)
                            (1) forced error mark
                              | "fatal error when running txn"
                              | github.com/cockroachdb/errors/withstack/*withstack.withStack::
                            Wraps: (2)
                            Wraps: (3) attached stack trace
                              -- stack trace:
                              | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*opStmt).executeStmt
                              |     github.com/cockroachdb/cockroach/pkg/workload/schemachange/operation_generator.go:3139
                              | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*schemaChangeWorker).runInTxn
                              |     github.com/cockroachdb/cockroach/pkg/workload/schemachange/schemachange.go:499
                              | github.com/cockroachdb/cockroach/pkg/workload/schemachange.(*schemaChangeWorker).run
                              |     github.com/cockroachdb/cockroach/pkg/workload/schemachange/schemachange.go:587
                              | pkg/ccl/testccl/workload/schemachange/schemachange_test_test.TestWorkload.TestWorkload.func4.func6
                              |     pkg/ccl/testccl/workload/schemachange/schemachange_test_test/pkg/ccl/testccl/workload/schemachange/schema_change_external_test.go:123
                              | golang.org/x/sync/errgroup.(*Group).Go.func1
                              |     golang.org/x/sync/errgroup/external/org_golang_x_sync/errgroup/errgroup.go:78
                              | runtime.goexit
                              |     src/runtime/asm_amd64.s:1695
                            Wraps: (4) ***UNEXPECTED ERROR; Received an unexpected execution error.
                            Wraps: (5) ERROR: relation "view_w1_126" (138): column name "crdb_internal_mvcc_timestamp" conflicts with a system column name (SQLSTATE 42701)
                            Error types: (1) *markers.withMark (2) *schemachange.ErrorState (3) *withstack.withStack (4) *errutil.withPrefix (5) *pgconn.PgError
            Test:           TestWorkload
Schema Workload Stats
Total Schema Statements Executed = 85
Total Schema Statements Succeeded = 85
Total Schema Statement Expected Failures = 0
Total Transactions Committed = 59
Total Transactions Rolled Back = 48
Total Transactions Executed = 107
    schema_change_external_test.go:99: backup, tracing data, and system table dumps in /var/lib/engflow/worker/work/3/exec/bazel-out/k8-fastbuild/testlogs/pkg/ccl/testccl/workload/schemachange/schemachange_test/run_19_of_25/test.outputs/logTestWorkload2300880375
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestWorkload2300880375
--- FAIL: TestWorkload (28.26s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #129462 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot P-2 T-sql-foundations branch-release-24.2.1-rc] - #128953 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed ["enum type does not exist" error code when setting a DEFAULT] [C-test-failure O-robot P-3 T-sql-foundations branch-release-23.2.10-rc]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 week ago

pkg/ccl/testccl/workload/schemachange/schemachange_test.TestWorkload failed on master @ 00edd9c46e7f3638846d0dc182a9dd50a0aa9cbc:

                              |   |     github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:152
                              |   | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*batchInfoCollector).Next
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/colflow/stats.go:126
                              |   | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*BatchFlowCoordinator).nextAdapter
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:246
                              |   | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:152
                              |   | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*BatchFlowCoordinator).next
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:250
                              |   | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*BatchFlowCoordinator).Run
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/colflow/flow_coordinator.go:282
                              |   | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlow).Run
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:320
                              |   | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:928
                              |   | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRun
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1990
                              |   | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRunAll.func3
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1704
                              |   | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRunAll
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:1707
                              |   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithDistSQLEngine
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:2427
                              |   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).dispatchToExecutionEngine
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:1974
                              |   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmtInOpenState
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:1181
                              |   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt.func1
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:146
                              |   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithProfiling
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:3436
                              |   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:145
                              |   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execPortal
                              |   |     github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:251
                              | Wraps: (2) single entry value with no default column id
                              | Error types: (1) *withstack.withStack (2) *errutil.leafError (SQLSTATE XX000)
                            Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *pgconn.PgError
            Test:           TestWorkload
Schema Workload Stats
Total Schema Statements Executed = 125
Total Schema Statements Succeeded = 125
Total Schema Statement Expected Failures = 0
Total Transactions Committed = 82
Total Transactions Rolled Back = 59
Total Transactions Executed = 141
    schema_change_external_test.go:99: backup, tracing data, and system table dumps in /var/lib/engflow/worker/work/1/exec/bazel-out/k8-fastbuild/testlogs/pkg/ccl/testccl/workload/schemachange/schemachange_test/run_14_of_25/test.outputs/logTestWorkload1273033940
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestWorkload1273033940
--- FAIL: TestWorkload (26.89s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #130283 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot T-sql-foundations branch-release-23.1 release-blocker] - #129462 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot P-2 T-sql-foundations branch-release-24.2.1-rc] - #128953 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed ["enum type does not exist" error code when setting a DEFAULT] [C-test-failure O-robot P-3 T-sql-foundations branch-release-23.2.10-rc]

This test on roachdash | Improve this report!

fqazi commented 6 days ago

Fixed via: https://github.com/cockroachdb/cockroach/pull/130351

cockroach-teamcity commented 6 days ago

pkg/ccl/testccl/workload/schemachange/schemachange_test.TestWorkload failed on master @ 7571549d8772a087ab4577d1e770e02582ba5877:

                              |   |     github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/store_send.go:193
                              |   | github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Stores).SendWithWriteBytes
                              |   |     github.com/cockroachdb/cockroach/pkg/kv/kvserver/pkg/kv/kvserver/stores.go:202
                              |   | github.com/cockroachdb/cockroach/pkg/server.(*Node).batchInternal
                              |   |     github.com/cockroachdb/cockroach/pkg/server/node.go:1635
                              |   | github.com/cockroachdb/cockroach/pkg/server.(*Node).Batch
                              |   |     github.com/cockroachdb/cockroach/pkg/server/node.go:1836
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.func1
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:718
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.ServerInterceptor.func12
                              |   |     github.com/cockroachdb/cockroach/pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:97
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:829
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func3
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:170
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:829
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.kvAuth.unaryInterceptor
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/auth.go:95
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:829
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1.1
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:137
                              |   | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr
                              |   |     github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:336
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:135
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func4
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:829
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.func2
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:728
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.NewContext.ClientInterceptor.func8
                              |   |     github.com/cockroachdb/cockroach/pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:228
                              |   | github.com/cockroachdb/cockroach/pkg/rpc.getChainUnaryInvoker.func1
                              |   |     github.com/cockroachdb/cockroach/pkg/rpc/pkg/rpc/context.go:913
                              | Wraps: (2) single entry value with no default column id
                              | Error types: (1) *withstack.withStack (2) *errutil.leafError (SQLSTATE XXUUU)
                            Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *pgconn.PgError
            Test:           TestWorkload
Schema Workload Stats
Total Schema Statements Executed = 39
Total Schema Statements Succeeded = 39
Total Schema Statement Expected Failures = 0
Total Transactions Committed = 27
Total Transactions Rolled Back = 19
Total Transactions Executed = 46
    schema_change_external_test.go:99: backup, tracing data, and system table dumps in /var/lib/engflow/worker/work/2/exec/bazel-out/k8-fastbuild/testlogs/pkg/ccl/testccl/workload/schemachange/schemachange_test/run_5_of_25/test.outputs/logTestWorkload650788707
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestWorkload650788707
--- FAIL: TestWorkload (15.74s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

- #130283 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot T-sql-foundations branch-release-23.1 release-blocker] - #129462 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed [C-test-failure O-robot P-2 T-sql-foundations branch-release-24.2.1-rc] - #128953 pkg/ccl/testccl/workload/schemachange/schemachange_test: TestWorkload failed ["enum type does not exist" error code when setting a DEFAULT] [C-test-failure O-robot P-3 T-sql-foundations branch-release-23.2.10-rc]

This test on roachdash | Improve this report!