Closed cockroach-teamcity closed 3 years ago
sql/logictest.TestLogic failed with artifacts on master @ bf6da8a40f6956bc88ee4a930ddc931f73dd5ce4:
=== CONT TestLogic
logic.go:3312: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic993746173
--- FAIL: TestLogic (1080.73s)
=== RUN TestLogic/fakedist-spec-planning/target_names
=== PAUSE TestLogic/fakedist-spec-planning/target_names
=== CONT TestLogic/fakedist-spec-planning/target_names
=== CONT TestLogic/fakedist-spec-planning/target_names
logic.go:2528:
testdata/logic_test/target_names:39: SELECT (pg_get_keywords()).word FROM t
expected success, but found
(XXUUU) failed to connect to n2 at 127.0.0.1:44747: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
nodedialer.go:169: in dial()
E210419 22:25:18.149024 11276971 sql/logictest/logic.go:3519 [-] 1169
E210419 22:25:18.149024 11276971 sql/logictest/logic.go:3519 [-] 1169 +pq: failed to connect to n2 at 127.0.0.1:44747: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
logic.go:2261:
pq: failed to connect to n2 at 127.0.0.1:44747: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
--- done: testdata/logic_test/target_names with config fakedist-spec-planning: 6 tests, 2 failures
E210419 22:25:18.149481 11276971 sql/logictest/logic.go:3544 [-] 1170 testdata/logic_test/target_names:44: too many errors encountered, skipping the rest of the input
logic.go:3087:
testdata/logic_test/target_names:44: error while processing
logic.go:3087: testdata/logic_test/target_names:44: too many errors encountered, skipping the rest of the input
--- FAIL: TestLogic/fakedist-spec-planning/target_names (1.41s)
=== RUN TestLogic/fakedist-spec-planning/cascade/OptDriven
=== CONT TestLogic/fakedist-spec-planning/cascade/OptDriven
logic.go:2528:
testdata/logic_test/cascade:15: SELECT * FROM child
expected success, but found
(XXUUU) failed to connect to n3 at 127.0.0.1:46129: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
nodedialer.go:169: in dial()
E210419 22:25:36.054415 12385639 sql/logictest/logic.go:3519 [-] 1198
E210419 22:25:36.054415 12385639 sql/logictest/logic.go:3519 [-] 1198 +pq: failed to connect to n3 at 127.0.0.1:46129: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
logic.go:2261:
pq: failed to connect to n3 at 127.0.0.1:46129: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
E210419 22:25:36.054706 12385639 sql/logictest/logic.go:3519 [-] 1199
E210419 22:25:36.054706 12385639 sql/logictest/logic.go:3519 [-] 1199 +testdata/logic_test/cascade:23: too many errors encountered, skipping the rest of the input
logic.go:1805:
testdata/logic_test/cascade:23: too many errors encountered, skipping the rest of the input
--- done: testdata/logic_test/cascade with config fakedist-spec-planning: 2 tests, 3 failures
--- done: testdata/logic_test/window with config fakedist-spec-planning: 397 tests, 0 failures
--- total progress: 131332 statements/queries
--- FAIL: TestLogic/fakedist-spec-planning/cascade/OptDriven (0.68s)
=== RUN TestLogic/fakedist-spec-planning/cascade
=== PAUSE TestLogic/fakedist-spec-planning/cascade
=== CONT TestLogic/fakedist-spec-planning/cascade
--- FAIL: TestLogic/fakedist-spec-planning/cascade (1.91s)
=== RUN TestLogic/fakedist-spec-planning
--- FAIL: TestLogic/fakedist-spec-planning (0.72s)
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ```
Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.1]
/cc @jordanlewis
sql/logictest.TestLogic failed with artifacts on master @ 1c103726de6994b26706244144d3d9f7dd6b1f58:
=== RUN TestLogic
test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic039425781
test_log_scope.go:74: use -show-logs to present logs inline
=== CONT TestLogic
logic.go:3312: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic039425781
--- FAIL: TestLogic (987.15s)
=== RUN TestLogic/fakedist-spec-planning
--- FAIL: TestLogic/fakedist-spec-planning (0.58s)
=== RUN TestLogic/fakedist-spec-planning/timestamp/timestamp_precision
=== CONT TestLogic/fakedist-spec-planning/timestamp/timestamp_precision
logic.go:2528:
testdata/logic_test/timestamp:100: SELECT * FROM timestamp_test ORDER BY id ASC
expected success, but found
(XXUUU) failed to connect to n3 at 127.0.0.1:40241: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
nodedialer.go:169: in dial()
E210419 22:59:32.858674 11450222 sql/logictest/logic.go:3519 [-] 1125
E210419 22:59:32.858674 11450222 sql/logictest/logic.go:3519 [-] 1125 +pq: failed to connect to n3 at 127.0.0.1:40241: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
logic.go:2261:
pq: failed to connect to n3 at 127.0.0.1:40241: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
E210419 22:59:32.859058 11450222 sql/logictest/logic.go:3519 [-] 1126
E210419 22:59:32.859058 11450222 sql/logictest/logic.go:3519 [-] 1126 +testdata/logic_test/timestamp:111: too many errors encountered, skipping the rest of the input
logic.go:1805:
testdata/logic_test/timestamp:111: too many errors encountered, skipping the rest of the input
--- done: testdata/logic_test/timestamp with config fakedist-spec-planning: 19 tests, 3 failures
--- FAIL: TestLogic/fakedist-spec-planning/timestamp/timestamp_precision (0.50s)
=== RUN TestLogic/fakedist-spec-planning/timestamp
=== PAUSE TestLogic/fakedist-spec-planning/timestamp
=== CONT TestLogic/fakedist-spec-planning/timestamp
--- FAIL: TestLogic/fakedist-spec-planning/timestamp (1.81s)
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ```
Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.1]
/cc @jordanlewis
sql/logictest.TestLogic failed with artifacts on master @ bea4754bdf607ab050a72b7e2a3fe9dca0289d34:
=== RUN TestLogic
test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic622773250
test_log_scope.go:74: use -show-logs to present logs inline
=== CONT TestLogic
logic.go:3321: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic622773250
--- FAIL: TestLogic (1030.15s)
=== RUN TestLogic/fakedist-spec-planning/computed
=== PAUSE TestLogic/fakedist-spec-planning/computed
=== CONT TestLogic/fakedist-spec-planning/computed
--- done: testdata/logic_test/conditional with config fakedist-spec-planning: 9 tests, 0 failures
=== CONT TestLogic/fakedist-spec-planning/computed
logic.go:2537:
testdata/logic_test/computed:72: SELECT c FROM with_no_column_refs
expected success, but found
(XXUUU) failed to connect to n3 at 127.0.0.1:38231: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
nodedialer.go:169: in dial()
E210420 12:38:46.870038 11122369 sql/logictest/logic.go:3528 [-] 1172
E210420 12:38:46.870038 11122369 sql/logictest/logic.go:3528 [-] 1172 +pq: failed to connect to n3 at 127.0.0.1:38231: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
logic.go:2270:
pq: failed to connect to n3 at 127.0.0.1:38231: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
--- done: testdata/logic_test/computed with config fakedist-spec-planning: 14 tests, 2 failures
E210420 12:38:46.870229 11122369 sql/logictest/logic.go:3553 [-] 1173 testdata/logic_test/computed:78: too many errors encountered, skipping the rest of the input
logic.go:3096:
testdata/logic_test/computed:78: error while processing
logic.go:3096: testdata/logic_test/computed:78: too many errors encountered, skipping the rest of the input
E210420 12:38:46.872919 11957392 sql/stats/automatic_stats.go:302 [-] 1174 failed to refresh stats: node unavailable; try another peer
--- FAIL: TestLogic/fakedist-spec-planning/computed (1.60s)
=== RUN TestLogic/fakedist-spec-planning
--- FAIL: TestLogic/fakedist-spec-planning (0.66s)
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ```
Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.1]
/cc @jordanlewis
sql/logictest.TestLogic failed with artifacts on master @ b4766e0bdb491de6020d5d2acaa3d20f9c339176:
=== RUN TestLogic
test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic692930075
test_log_scope.go:74: use -show-logs to present logs inline
=== CONT TestLogic
logic.go:3350: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic692930075
--- FAIL: TestLogic (794.20s)
=== RUN TestLogic/fakedist-vec-off/vectorize_unsupported
=== PAUSE TestLogic/fakedist-vec-off/vectorize_unsupported
=== CONT TestLogic/fakedist-vec-off/vectorize_unsupported
--- done: testdata/logic_test/where with config fakedist-vec-off: 21 tests, 0 failures
--- done: testdata/logic_test/inverted_index with config fakedist-vec-off: 264 tests, 0 failures
=== CONT TestLogic/fakedist-vec-off/vectorize_unsupported
logic.go:2539:
testdata/logic_test/vectorize_unsupported:28: select (x/1) from t39417
expected success, but found
(XXUUU) failed to connect to n2 at 127.0.0.1:35759: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
nodedialer.go:169: in dial()
E210422 20:37:40.337643 5095761 sql/logictest/logic.go:3557 [-] 587
E210422 20:37:40.337643 5095761 sql/logictest/logic.go:3557 [-] 587 +pq: failed to connect to n2 at 127.0.0.1:35759: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
logic.go:2272:
pq: failed to connect to n2 at 127.0.0.1:35759: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
--- done: testdata/logic_test/vectorize_unsupported with config fakedist-vec-off: 6 tests, 2 failures
E210422 20:37:40.346935 5095761 sql/logictest/logic.go:3582 [-] 588 testdata/logic_test/vectorize_unsupported:33: too many errors encountered, skipping the rest of the input
logic.go:3125:
testdata/logic_test/vectorize_unsupported:33: error while processing
logic.go:3125: testdata/logic_test/vectorize_unsupported:33: too many errors encountered, skipping the rest of the input
--- FAIL: TestLogic/fakedist-vec-off/vectorize_unsupported (1.83s)
=== RUN TestLogic/fakedist-vec-off
--- FAIL: TestLogic/fakedist-vec-off (0.60s)
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ```
Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.1]
/cc @jordanlewis
sql/logictest.TestLogic failed with artifacts on master @ 924935cc8ee14ff0c6d23aedd2a12673bf864808:
=== RUN TestLogic
test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic599276416
test_log_scope.go:74: use -show-logs to present logs inline
=== CONT TestLogic
logic.go:3350: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic599276416
--- FAIL: TestLogic (816.45s)
=== RUN TestLogic/fakedist/serial/serial_rowid
=== CONT TestLogic/fakedist/serial/serial_rowid
logic.go:2539:
testdata/logic_test/serial:31: SELECT count(DISTINCT a), count(DISTINCT b), count(DISTINCT c) FROM serial
expected success, but found
(XXUUU) failed to connect to n2 at 127.0.0.1:35655: initial connection heartbeat failed: rpc error: code = Unavailable desc = timed out waiting for server handshake
nodedialer.go:169: in dial()
E210424 06:04:35.253726 4748781 sql/logictest/logic.go:3557 [-] 533
E210424 06:04:35.253726 4748781 sql/logictest/logic.go:3557 [-] 533 +pq: failed to connect to n2 at 127.0.0.1:35655: initial connection heartbeat failed: rpc error: code = Unavailable desc = timed out waiting for server handshake
logic.go:2272:
pq: failed to connect to n2 at 127.0.0.1:35655: initial connection heartbeat failed: rpc error: code = Unavailable desc = timed out waiting for server handshake
E210424 06:04:35.254062 4748781 sql/logictest/logic.go:3557 [-] 534
E210424 06:04:35.254062 4748781 sql/logictest/logic.go:3557 [-] 534 +testdata/logic_test/serial:36: too many errors encountered, skipping the rest of the input
logic.go:1816:
testdata/logic_test/serial:36: too many errors encountered, skipping the rest of the input
--- done: testdata/logic_test/serial with config fakedist: 5 tests, 3 failures
--- FAIL: TestLogic/fakedist/serial/serial_rowid (0.31s)
=== RUN TestLogic/fakedist/serial
=== PAUSE TestLogic/fakedist/serial
=== CONT TestLogic/fakedist/serial
--- FAIL: TestLogic/fakedist/serial (1.64s)
=== RUN TestLogic/fakedist
--- FAIL: TestLogic/fakedist (0.58s)
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ```
Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.1]
/cc @jordanlewis
sql/logictest.TestLogic failed with artifacts on master @ 05880958efb6fda9ab80ed16495d98922de3fbf6:
Fatal error:
panic: expected 2 components to be closed, but found that only 0 were
Stack:
goroutine 9453985 [running]:
github.com/cockroachdb/cockroach/pkg/sql/colexecerror.InternalError(...)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:192
github.com/cockroachdb/cockroach/pkg/sql/colflow.(*vectorizedFlow).Cleanup(0xc146920aa0, 0x51d5e40, 0xc037fcf240)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:285 +0x505
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run.func8()
/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:373 +0x82
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithDistSQLEngine(0xc0c811c600, 0x51f4140, 0xc05fff0318, 0xc0c811cae0, 0x3, 0x5243ee0, 0xc08e468c00, 0xc109496401, 0xc0b0f19078, 0x0, ...)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:1055 +0x4cc
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).dispatchToExecutionEngine(0xc0c811c600, 0x51f4140, 0xc05fff0318, 0xc0c811cae0, 0x5243ee0, 0xc08e468c00, 0x0, 0x0)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:915 +0x75b
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmtInOpenState(0xc0c811c600, 0x51f4140, 0xc05fff0318, 0x5200700, 0xc02c975f90, 0xc0979d1bc0, 0x3c, 0x0, 0x0, 0x0, ...)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:659 +0xf86
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt(0xc0c811c600, 0x51d5e40, 0xc0ce4d52c0, 0x5200700, 0xc02c975f90, 0xc0979d1bc0, 0x3c, 0x0, 0x0, 0x0, ...)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:122 +0xb14
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd.func1(0x5200700, 0xc02c975f90, 0xc0979d1bc0, 0x3c, 0x0, 0x0, 0xad5db37, 0xed8193acf, 0x0, 0xad5db37, ...)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1527 +0x225
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd(0xc0c811c600, 0x51d5e40, 0xc0ce4d52c0, 0x0, 0x0)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1529 +0x1457
github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run(0xc0c811c600, 0x51d5f00, 0xc08e468870, 0xc081bf7e00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1450 +0x228
github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).initConnEx.func1(0xc0c811c600, 0x51d5f00, 0xc08e468870, 0xc0b0f18960, 0xc08e4689c0, 0x5188200, 0xc0ce4d5140, 0xc02ad0a630, 0xc17f462d20)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:202 +0x65
created by github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).initConnEx
/go/src/github.com/cockroachdb/cockroach/pkg/sql/internal.go:201 +0x2f2
``` === RUN TestLogic/fakedist-disk/serial/serial_virtual_sequence E210426 22:50:55.106794 9444974 jobs/adopt.go:260 [n1] 954 job 653417037656260609: adoption completed with error failed to ingest index entries during backfill: duplicate key value violates unique constraint "bar" ```
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ```
Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.1]
/cc @jordanlewis
sql/logictest.TestLogic failed with artifacts on master @ 588d9d7ad7618784ee23a5e5f440a9d9d36bb0d5:
=== RUN TestLogic/fakedist-disk/schema_change_in_txn/rollback_mutations
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ```
Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.1]
/cc @jordanlewis
sql/logictest.TestLogic failed with artifacts on master @ 29305bfb08116f38e10ca090347eea170899b919:
=== RUN TestLogic
test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic555034003
test_log_scope.go:74: use -show-logs to present logs inline
=== CONT TestLogic
logic.go:3351: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic555034003
--- FAIL: TestLogic (817.89s)
=== RUN TestLogic/fakedist/zigzag_join
=== PAUSE TestLogic/fakedist/zigzag_join
=== CONT TestLogic/fakedist/zigzag_join
=== CONT TestLogic/fakedist/zigzag_join
logic.go:2540:
testdata/logic_test/zigzag_join:38: SELECT n,a,b FROM a WHERE a = 4 AND b = 1
expected success, but found
(XXUUU) failed to connect to n3 at 127.0.0.1:44899: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
nodedialer.go:169: in dial()
E210427 21:45:54.071236 3454388 sql/logictest/logic.go:3558 [-] 479
E210427 21:45:54.071236 3454388 sql/logictest/logic.go:3558 [-] 479 +pq: failed to connect to n3 at 127.0.0.1:44899: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
logic.go:2273:
pq: failed to connect to n3 at 127.0.0.1:44899: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
--- done: testdata/logic_test/zigzag_join with config fakedist: 8 tests, 2 failures
E210427 21:45:54.072134 3454388 sql/logictest/logic.go:3583 [-] 480 testdata/logic_test/zigzag_join:43: too many errors encountered, skipping the rest of the input
logic.go:3126:
testdata/logic_test/zigzag_join:43: error while processing
logic.go:3126: testdata/logic_test/zigzag_join:43: too many errors encountered, skipping the rest of the input
--- FAIL: TestLogic/fakedist/zigzag_join (1.81s)
=== RUN TestLogic/fakedist
--- FAIL: TestLogic/fakedist (0.75s)
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ```
Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.1]
/cc @jordanlewis
sql/logictest.TestLogic failed with artifacts on master @ f3286c1e1b8d2d44f0b5c92460c0d5acbfdd1610:
=== RUN TestLogic
test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic036720330
test_log_scope.go:74: use -show-logs to present logs inline
=== CONT TestLogic
logic.go:3351: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic036720330
--- FAIL: TestLogic (828.11s)
=== RUN TestLogic/5node-metadata/distsql_numtables
=== PAUSE TestLogic/5node-metadata/distsql_numtables
=== CONT TestLogic/5node-metadata/distsql_numtables
=== CONT TestLogic/5node-metadata/distsql_numtables
logic.go:2540:
testdata/logic_test/distsql_numtables:21:
expected success, but found
(XXUUU) while carrying out changes [{ADD_VOTER n4,s4}]: failed to connect to n4 at 127.0.0.1:35873: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
nodedialer.go:169: in dial()
--- done: testdata/logic_test/distsql_numtables with config 5node-metadata: 4 tests, 1 failures
E210427 22:20:54.582791 12005955 sql/logictest/logic.go:3583 [-] 1219 pq: while carrying out changes [{ADD_VOTER n4,s4}]: failed to connect to n4 at 127.0.0.1:35873: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
logic.go:3126:
testdata/logic_test/distsql_numtables:21: error while processing
logic.go:3126: pq: while carrying out changes [{ADD_VOTER n4,s4}]: failed to connect to n4 at 127.0.0.1:35873: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
--- done: testdata/logic_test/contention_event with config 5node-metadata: 12 tests, 0 failures
--- total progress: 140737 statements/queries
--- FAIL: TestLogic/5node-metadata/distsql_numtables (2.22s)
=== RUN TestLogic/5node-metadata
--- FAIL: TestLogic/5node-metadata (0.00s)
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ```
Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.1]
/cc @jordanlewis
sql/logictest.TestLogic failed with artifacts on master @ 171b1bd0971171214c203ad615e40a651b71848a:
=== RUN TestLogic
test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic231064638
test_log_scope.go:74: use -show-logs to present logs inline
=== CONT TestLogic
logic.go:3351: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic231064638
--- FAIL: TestLogic (800.82s)
=== RUN TestLogic/fakedist-disk/schema_change_in_txn
=== PAUSE TestLogic/fakedist-disk/schema_change_in_txn
=== CONT TestLogic/fakedist-disk/schema_change_in_txn
=== CONT TestLogic/fakedist-disk/schema_change_in_txn
testcluster.go:135: condition failed to evaluate within 45s: unexpectedly found 1 active spans:
0.000ms 0.000ms === operation:colbatchscan
goroutine 8364100 [running]:
runtime/debug.Stack(0xc00356b860, 0x5152500, 0xc12816d700)
/usr/local/go/src/runtime/debug/stack.go:24 +0x9f
github.com/cockroachdb/cockroach/pkg/testutils.SucceedsSoon(0x53233a0, 0xc008bd1080, 0xc00356b860)
/go/src/github.com/cockroachdb/cockroach/pkg/testutils/soon.go:36 +0x6d
github.com/cockroachdb/cockroach/pkg/testutils/testcluster.(*TestCluster).stopServers(0xc087166000, 0x51de520, 0xc000072128)
/go/src/github.com/cockroachdb/cockroach/pkg/testutils/testcluster/testcluster.go:135 +0x23d
github.com/cockroachdb/cockroach/pkg/testutils/testcluster.(*TestCluster).Start.func2()
/go/src/github.com/cockroachdb/cockroach/pkg/testutils/testcluster/testcluster.go:335 +0x45
github.com/cockroachdb/cockroach/pkg/util/stop.CloserFn.Close(0xc0792f12e0)
/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:110 +0x25
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).Stop(0xc06c0fb780, 0x51de520, 0xc000072128)
/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:479 +0x22f
github.com/cockroachdb/cockroach/pkg/sql/logictest.(*logicTest).shutdownCluster(0xc088faa800)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/logictest/logic.go:1602 +0x1da
github.com/cockroachdb/cockroach/pkg/sql/logictest.(*logicTest).close(0xc088faa800)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/logictest/logic.go:1237 +0x3d
github.com/cockroachdb/cockroach/pkg/sql/logictest.(*logicTest).runFile(0xc088faa800, 0xc000c8f200, 0x28, 0x450b498, 0xd, 0x3, 0x1, 0x44f2451, 0x2, 0x0, ...)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/logictest/logic.go:3132 +0x189
github.com/cockroachdb/cockroach/pkg/sql/logictest.RunLogicTestWithDefaultConfig.func1.1(0xc008bd1080)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/logictest/logic.go:3324 +0x2de
testing.tRunner(0xc008bd1080, 0xc01d91f440)
/usr/local/go/src/testing/testing.go:1123 +0xef
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1168 +0x2b3
--- progress: testdata/logic_test/tenant: 25 statements/queries
--- done: testdata/logic_test/tenant with config fakedist-disk: 27 tests, 0 failures
--- total progress: 122871 statements/queries
--- FAIL: TestLogic/fakedist-disk/schema_change_in_txn (65.26s)
=== RUN TestLogic/fakedist-disk
--- FAIL: TestLogic/fakedist-disk (0.65s)
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ```
Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.1]
/cc @jordanlewis
sql/logictest.TestLogic failed with artifacts on master @ 112d0c111c2e9bd54cd13d40ad353cc4b286016e:
=== RUN TestLogic
test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic265796156
test_log_scope.go:74: use -show-logs to present logs inline
=== CONT TestLogic
logic.go:3351: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic265796156
--- FAIL: TestLogic (1011.02s)
=== RUN TestLogic/5node
--- FAIL: TestLogic/5node (0.00s)
=== RUN TestLogic/5node/distsql_distinct_on
=== PAUSE TestLogic/5node/distsql_distinct_on
=== CONT TestLogic/5node/distsql_distinct_on
=== CONT TestLogic/5node/distsql_distinct_on
logic.go:2540:
testdata/logic_test/distsql_distinct_on:40:
expected success, but found
(XXUUU) while carrying out changes [{ADD_VOTER n3,s3}]: failed to connect to n3 at 127.0.0.1:36903: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
nodedialer.go:169: in dial()
--- done: testdata/logic_test/distsql_distinct_on with config 5node: 5 tests, 1 failures
E210428 11:16:37.957233 13281632 sql/logictest/logic.go:3583 [-] 1274 pq: while carrying out changes [{ADD_VOTER n3,s3}]: failed to connect to n3 at 127.0.0.1:36903: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
logic.go:3126:
testdata/logic_test/distsql_distinct_on:40: error while processing
logic.go:3126: pq: while carrying out changes [{ADD_VOTER n3,s3}]: failed to connect to n3 at 127.0.0.1:36903: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
--- FAIL: TestLogic/5node/distsql_distinct_on (2.22s)
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ```
Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.1]
/cc @jordanlewis
sql/logictest.TestLogic failed with artifacts on master @ 7910db27b63ad68d2b43f618976ef03da5451ed2:
=== RUN TestLogic
test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic952021909
test_log_scope.go:74: use -show-logs to present logs inline
=== CONT TestLogic
logic.go:3351: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic952021909
--- FAIL: TestLogic (931.44s)
=== RUN TestLogic/fakedist-spec-planning/schema_change_in_txn
=== PAUSE TestLogic/fakedist-spec-planning/schema_change_in_txn
=== CONT TestLogic/fakedist-spec-planning/schema_change_in_txn
--- progress: testdata/logic_test/join: 102 statements/queries
--- done: testdata/logic_test/select with config fakedist-spec-planning: 162 tests, 0 failures
=== CONT TestLogic/fakedist-spec-planning/schema_change_in_txn
testcluster.go:135: condition failed to evaluate within 45s: unexpectedly found 1 active spans:
0.000ms 0.000ms === operation:colbatchscan
goroutine 10131596 [running]:
runtime/debug.Stack(0xc07875b860, 0x5152860, 0xc3015f0fe0)
/usr/local/go/src/runtime/debug/stack.go:24 +0x9f
github.com/cockroachdb/cockroach/pkg/testutils.SucceedsSoon(0x5323840, 0xc03e975c80, 0xc07875b860)
/go/src/github.com/cockroachdb/cockroach/pkg/testutils/soon.go:36 +0x6d
github.com/cockroachdb/cockroach/pkg/testutils/testcluster.(*TestCluster).stopServers(0xc26c7c2480, 0x51de8c0, 0xc00019e018)
/go/src/github.com/cockroachdb/cockroach/pkg/testutils/testcluster/testcluster.go:135 +0x23d
github.com/cockroachdb/cockroach/pkg/testutils/testcluster.(*TestCluster).Start.func2()
/go/src/github.com/cockroachdb/cockroach/pkg/testutils/testcluster/testcluster.go:335 +0x45
github.com/cockroachdb/cockroach/pkg/util/stop.CloserFn.Close(0xc2fdab6c50)
/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:110 +0x25
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).Stop(0xc0365ad280, 0x51de8c0, 0xc00019e018)
/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:479 +0x22f
github.com/cockroachdb/cockroach/pkg/sql/logictest.(*logicTest).shutdownCluster(0xc239594a00)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/logictest/logic.go:1602 +0x1da
github.com/cockroachdb/cockroach/pkg/sql/logictest.(*logicTest).close(0xc239594a00)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/logictest/logic.go:1237 +0x3d
github.com/cockroachdb/cockroach/pkg/sql/logictest.(*logicTest).runFile(0xc239594a00, 0xc000ea5230, 0x28, 0x452aca3, 0x16, 0x3, 0x1, 0x44f25d1, 0x2, 0x0, ...)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/logictest/logic.go:3132 +0x189
github.com/cockroachdb/cockroach/pkg/sql/logictest.RunLogicTestWithDefaultConfig.func1.1(0xc03e975c80)
/go/src/github.com/cockroachdb/cockroach/pkg/sql/logictest/logic.go:3324 +0x2de
testing.tRunner(0xc03e975c80, 0xc1b4c981c0)
/usr/local/go/src/testing/testing.go:1123 +0xef
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1168 +0x2b3
--- FAIL: TestLogic/fakedist-spec-planning/schema_change_in_txn (65.28s)
=== RUN TestLogic/fakedist-spec-planning
--- FAIL: TestLogic/fakedist-spec-planning (0.58s)
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ```
Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.1]
/cc @jordanlewis
sql/logictest.TestLogic failed with artifacts on master @ 852e22a2ad78b1a19bd4f504f7079274c0c95fe5:
=== RUN TestLogic
test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic570632428
test_log_scope.go:74: use -show-logs to present logs inline
=== CONT TestLogic
logic.go:3351: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic570632428
--- FAIL: TestLogic (1167.38s)
=== RUN TestLogic/fakedist-spec-planning/poison_after_push
=== PAUSE TestLogic/fakedist-spec-planning/poison_after_push
=== CONT TestLogic/fakedist-spec-planning/poison_after_push
--- progress: testdata/logic_test/fk: 252 statements/queries
--- done: testdata/logic_test/rename_constraint with config fakedist-spec-planning: 15 tests, 0 failures
=== CONT TestLogic/fakedist-spec-planning/poison_after_push
logic.go:2540:
testdata/logic_test/poison_after_push:35: SELECT * FROM t
expected success, but found
(XXUUU) failed to connect to n2 at 127.0.0.1:33935: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
nodedialer.go:169: in dial()
E210428 18:16:09.720477 11599440 sql/logictest/logic.go:3558 [-] 1206
E210428 18:16:09.720477 11599440 sql/logictest/logic.go:3558 [-] 1206 +pq: failed to connect to n2 at 127.0.0.1:33935: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
logic.go:2273:
pq: failed to connect to n2 at 127.0.0.1:33935: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
--- done: testdata/logic_test/poison_after_push with config fakedist-spec-planning: 7 tests, 2 failures
E210428 18:16:09.723213 11599440 sql/logictest/logic.go:3583 [-] 1207 testdata/logic_test/poison_after_push:40: too many errors encountered, skipping the rest of the input
logic.go:3126:
testdata/logic_test/poison_after_push:40: error while processing
logic.go:3126: testdata/logic_test/poison_after_push:40: too many errors encountered, skipping the rest of the input
--- FAIL: TestLogic/fakedist-spec-planning/poison_after_push (1.92s)
=== RUN TestLogic/fakedist-spec-planning
--- FAIL: TestLogic/fakedist-spec-planning (0.67s)
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ```
Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.1]
/cc @jordanlewis
sql/logictest.TestLogic failed with artifacts on master @ ea6bcca8816a2705e46481eeb7543b16bb7c258c:
=== RUN TestLogic
test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic134511204
test_log_scope.go:74: use -show-logs to present logs inline
=== CONT TestLogic
logic.go:3351: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic134511204
--- FAIL: TestLogic (840.65s)
=== RUN TestLogic/fakedist/poison_after_push
=== PAUSE TestLogic/fakedist/poison_after_push
=== CONT TestLogic/fakedist/poison_after_push
=== CONT TestLogic/fakedist/poison_after_push
logic.go:2540:
testdata/logic_test/poison_after_push:35: SELECT * FROM t
expected success, but found
(XXUUU) failed to connect to n2 at 127.0.0.1:43475: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
nodedialer.go:169: in dial()
E210429 07:43:04.114254 3494957 sql/logictest/logic.go:3558 [-] 511
E210429 07:43:04.114254 3494957 sql/logictest/logic.go:3558 [-] 511 +pq: failed to connect to n2 at 127.0.0.1:43475: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
logic.go:2273:
pq: failed to connect to n2 at 127.0.0.1:43475: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
--- done: testdata/logic_test/poison_after_push with config fakedist: 7 tests, 2 failures
E210429 07:43:04.115097 3494957 sql/logictest/logic.go:3583 [-] 512 testdata/logic_test/poison_after_push:40: too many errors encountered, skipping the rest of the input
logic.go:3126:
testdata/logic_test/poison_after_push:40: error while processing
logic.go:3126: testdata/logic_test/poison_after_push:40: too many errors encountered, skipping the rest of the input
E210429 07:43:04.118965 4588676 sql/stats/automatic_stats.go:302 [-] 513 failed to refresh stats: node unavailable; try another peer
--- FAIL: TestLogic/fakedist/poison_after_push (1.42s)
=== RUN TestLogic/fakedist/pg_extension
=== PAUSE TestLogic/fakedist/pg_extension
=== CONT TestLogic/fakedist/pg_extension
E210429 07:43:02.638568 4568557 jobs/adopt.go:260 [n1] 500 job 654087889146118145: adoption completed with error could not remove enum value "hello" as it is being used by "use_greetings2" in row: k=1, v='hello'
--- done: testdata/logic_test/orms with config fakedist: 21 tests, 0 failures
=== CONT TestLogic/fakedist/pg_extension
logic.go:3130:
testdata/logic_test/pg_extension:55: error while processing
logic.go:3130: test was successful but validation upon completion failed: failed to test for descriptor JSON round-trip: pq: failed to connect to n3 at 127.0.0.1:44877: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
E210429 07:43:04.106818 4588518 sql/stats/automatic_stats.go:302 [-] 509 failed to refresh stats: node unavailable; try another peer
E210429 07:43:04.113834 3494957 sql/logictest/logic.go:3571 [-] 510
E210429 07:43:04.113834 3494957 sql/logictest/logic.go:3571 [-] 510 +testdata/logic_test/poison_after_push:35: SELECT * FROM t
E210429 07:43:04.113834 3494957 sql/logictest/logic.go:3571 [-] 510 +expected success, but found
E210429 07:43:04.113834 3494957 sql/logictest/logic.go:3571 [-] 510 +(XXUUU) failed to connect to n2 at 127.0.0.1:43475: initial connection heartbeat failed: rpc error: code = Unavailable desc = connection error: desc = "transport: authentication handshake failed: context deadline exceeded"
E210429 07:43:04.113834 3494957 sql/logictest/logic.go:3571 [-] 510 +nodedialer.go:169: in dial()
--- FAIL: TestLogic/fakedist/pg_extension (1.57s)
=== RUN TestLogic/fakedist
--- FAIL: TestLogic/fakedist (0.76s)
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ```
Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.1]
/cc @jordanlewis
sql/logictest.TestLogic failed with artifacts on master @ 0604c999494816236a1ed8660405144f071e655b:
=== RUN TestLogic
test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic738526644
test_log_scope.go:74: use -show-logs to present logs inline
=== CONT TestLogic
logic.go:3355: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic738526644
--- FAIL: TestLogic (1221.08s)
=== RUN TestLogic/local-vec-off/inflight_trace_spans
=== PAUSE TestLogic/local-vec-off/inflight_trace_spans
=== CONT TestLogic/local-vec-off/inflight_trace_spans
=== CONT TestLogic/local-vec-off/inflight_trace_spans
logic.go:2317: let $curr_trace_id = 1139691570243500368
E210624 19:44:03.657660 1390457 sql/logictest/logic.go:3563 [-] 170
E210624 19:44:03.657660 1390457 sql/logictest/logic.go:3563 [-] 170 +
E210624 19:44:03.657660 1390457 sql/logictest/logic.go:3563 [-] 170 +testdata/logic_test/inflight_trace_spans:44: SELECT count(*) = 0
E210624 19:44:03.657660 1390457 sql/logictest/logic.go:3563 [-] 170 + FROM current_trace_spans
E210624 19:44:03.657660 1390457 sql/logictest/logic.go:3563 [-] 170 +expected:
E210624 19:44:03.657660 1390457 sql/logictest/logic.go:3563 [-] 170 + true
E210624 19:44:03.657660 1390457 sql/logictest/logic.go:3563 [-] 170 +but found (query options: "") :
E210624 19:44:03.657660 1390457 sql/logictest/logic.go:3563 [-] 170 + false
logic.go:2276:
testdata/logic_test/inflight_trace_spans:44: SELECT count(*) = 0
FROM current_trace_spans
expected:
true
but found (query options: "") :
false
--- done: testdata/logic_test/inflight_trace_spans with config local-vec-off: 9 tests, 1 failures
--- FAIL: TestLogic/local-vec-off/inflight_trace_spans (3.13s)
=== RUN TestLogic/local-vec-off
--- FAIL: TestLogic/local-vec-off (0.93s)
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ```
Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot T-sql-schema branch-release-20.1]
/cc @cockroachdb/sql-queries jordanlewis
sql/logictest.TestLogic failed with artifacts on master @ 5989de86b019c154076e42eb0cac1a55ac5c0be3:
=== RUN TestLogic
test_log_scope.go:73: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic769877379
test_log_scope.go:74: use -show-logs to present logs inline
=== CONT TestLogic
logic.go:3362: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic769877379
--- FAIL: TestLogic (1369.21s)
=== RUN TestLogic/local/inflight_trace_spans
=== PAUSE TestLogic/local/inflight_trace_spans
=== CONT TestLogic/local/inflight_trace_spans
--- done: testdata/logic_test/int_size with config local: 32 tests, 0 failures
--- progress: testdata/logic_test/vectorize: 155 statements/queries
--- progress: testdata/logic_test/role: 263 statements/queries
=== CONT TestLogic/local/inflight_trace_spans
logic.go:2324: let $curr_trace_id = 9111394917085194647
=== CONT TestLogic/local/inflight_trace_spans
logic.go:2283:
testdata/logic_test/inflight_trace_spans:44: SELECT count(*) = 0
FROM current_trace_spans
expected:
true
but found (query options: "") :
false
--- done: testdata/logic_test/inflight_trace_spans with config local: 9 tests, 1 failures
--- FAIL: TestLogic/local/inflight_trace_spans (3.20s)
=== RUN TestLogic/local
--- FAIL: TestLogic/local (1.29s)
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ```
Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot T-sql-schema branch-release-20.1]
/cc @cockroachdb/sql-queries jordanlewis
sql/logictest.TestLogic failed with artifacts on master @ e723a85739a2e1b514f5de3221a2177a3a37d0e6:
concat
expected:
count_1 count_lower count_upper concat
3 3 3 aaa
3 3 3 bbb
but found (query options: "colnames") :
count_1 count_lower count_upper concat
3 3 3
3 3 3 bbb
E210812 05:33:45.323018 10152167 sql/logictest/logic.go:3566 [-] 939
E210812 05:33:45.323018 10152167 sql/logictest/logic.go:3566 [-] 939 +testdata/logic_test/aggregate:3134: too many errors encountered, skipping the rest of the input
logic.go:1818:
testdata/logic_test/aggregate:3134: too many errors encountered, skipping the rest of the input
--- done: testdata/logic_test/aggregate with config fakedist-disk: 499 tests, 2 failures
--- FAIL: TestLogic/fakedist-disk/aggregate/string_agg (1.34s)
=== RUN TestLogic/fakedist-disk/aggregate
=== PAUSE TestLogic/fakedist-disk/aggregate
=== CONT TestLogic/fakedist-disk/aggregate
--- FAIL: TestLogic/fakedist-disk/aggregate (12.51s)
=== RUN TestLogic/fakedist-disk/window
=== PAUSE TestLogic/fakedist-disk/window
=== CONT TestLogic/fakedist-disk/window
=== CONT TestLogic/fakedist-disk/window
logic.go:2274:
testdata/logic_test/window:385: SELECT k, concat_agg(s) OVER (PARTITION BY v ORDER BY w, k) FROM kv ORDER BY 1
expected:
1 ba
3 Aa
5 NULL
6 bab
7 b
8 A
but found (query options: "") :
1
3 Aa
5 NULL
6
7
8 A
--- done: testdata/logic_test/window with config fakedist-disk: 56 tests, 1 failures
E210812 05:34:13.660944 9811824 sql/logictest/logic.go:3591 [-] 990 testdata/logic_test/window:395: too many errors encountered, skipping the rest of the input
logic.go:3132:
testdata/logic_test/window:395: error while processing
logic.go:3132: testdata/logic_test/window:395: too many errors encountered, skipping the rest of the input
--- FAIL: TestLogic/fakedist-disk/window (2.27s)
=== RUN TestLogic/fakedist-disk
--- FAIL: TestLogic/fakedist-disk (0.63s)
To reproduce, try: ```bash make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ``` Parameters in this failure: - GOFLAGS=-json
- #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2] - #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1] - #58269 sql/logictest: TestLogic failed [C-test-failure O-robot T-sql-schema branch-release-20.1]
/cc @cockroachdb/sql-queries jordanlewis
I was able to reproduce something related to the last hit here with fakedist-disk
config after one and a half hours on the gceworker:
=== CONT TestLogic/fakedist-disk/window
logic.go:2541:
testdata/logic_test/window:2119: SELECT product_name, price, nth_value(product_name, 2) OVER w AS second FROM products WHERE price = 200 OR price = 700 WINDOW w as (PARTITION BY price ORDER BY product_name RANGE BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) ORDER BY price, product_name
expected success, but found
(XX000) internal error: unexpectedly found decreasing offsets: [0 7 0 9]
bytes.go:80: in AssertOffsetsAreNonDecreasing()
DETAIL: stack trace:
github.com/cockroachdb/cockroach/pkg/col/coldata/bytes.go:80: AssertOffsetsAreNonDecreasing()
github.com/cockroachdb/cockroach/pkg/col/coldata/bytes.go:547: AssertOffsetsAreNonDecreasing()
github.com/cockroachdb/cockroach/pkg/sql/colexec/invariants_checker.go:87: Next()
github.com/cockroachdb/cockroach/pkg/sql/colflow/stats.go:98: Next()
github.com/cockroachdb/cockroach/pkg/sql/colexec/sort.go:146: spool()
github.com/cockroachdb/cockroach/pkg/sql/colexec/sort.go:283: Next()
github.com/cockroachdb/cockroach/pkg/sql/colexec/disk_spiller.go:197: func1()
github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:91: CatchVectorizedRuntimeError()
github.com/cockroachdb/cockroach/pkg/sql/colexec/disk_spiller.go:195: Next()
github.com/cockroachdb/cockroach/pkg/sql/colexec/invariants_checker.go:79: Next()
github.com/cockroachdb/cockroach/pkg/sql/colflow/stats.go:98: Next()
github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils/deselector.go:63: Next()
github.com/cockroachdb/cockroach/pkg/sql/colflow/colrpc/outbox.go:282: func1()
github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:91: CatchVectorizedRuntimeError()
github.com/cockroachdb/cockroach/pkg/sql/colflow/colrpc/outbox.go:273: sendBatches()
github.com/cockroachdb/cockroach/pkg/sql/colflow/colrpc/outbox.go:407: runWithStream()
github.com/cockroachdb/cockroach/pkg/sql/colflow/colrpc/outbox.go:216: Run()
github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:729: func1()
github.com/cockroachdb/cockroach/pkg/sql/colflow/vectorized_flow.go:1320: 1()
The expected output for the query is
Microsoft Lumia 200.00 Samsung
Samsung 200.00 NULL
Lenovo Thinkpad 700.00 Sony VAIO
Sony VAIO 700.00 iPad
iPad 700.00 NULL
It seems that the offsets are about the third column.
Guessing is that we're forgetting to call coldata.Batch.SetLength()
somewhere, and it looks like the window functions might be to blame, but I didn't immediately find where. @DrewKimball curious if you have any insight. (Note that in the comment above both aggregate
and window
logic test files failed, but there are some window function in aggregate
.)
I'm also not sure how/why fakedist-disk
is the failing config.
I don't see how we could be missing a SetLength
, since it's always called immediately before outputting a batch. I think it might actually be a problem with Truncate
- it can actually increase maxSetLength
, meaning it increases the index from which we update offsets to be non-decreasing. I'll try and figure out exactly how that might happen.
Hm, it looks like the one case where SpillingQueue.Enqueue
doesn't call SetLength
is when we spill immediately to disk and there is no selection vector. In that case, it should be possible to call Truncate
with a larger index than maxSetIndex
when there are nulls in the output. I'll see if I can write a query that reliably reproduces. We'd have to hit batch boundaries just right, so it might be tricky.
(sql/logictest).TestLogic failed on master@557346d5fc755db5f64d3e198e47be1f53a4d390:
More
Parameters: - GOFLAGS=-json ``` make stressrace TESTS=TestLogic PKG=./pkg/sql/logictest TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1 ``` Related: - #58269 sql/logictest: TestLogic failed [C-test-failure](https://api.github.com/repos/cockroachdb/cockroach/labels/C-test-failure) [O-robot](https://api.github.com/repos/cockroachdb/cockroach/labels/O-robot) [branch-release-20.1](https://api.github.com/repos/cockroachdb/cockroach/labels/branch-release-20.1) [See this test on roachdash](https://roachdash.crdb.dev/?filter=status%3Aopen+t%3A.%2ATestLogic.%2A&sort=title&restgroup=false&display=lastcommented+project) powered by [pkg/cmd/internal/issues](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)