Open cockroach-teamcity opened 2 weeks ago
from test.log:
bash: line 8: 39888 Aborted (core dumped) bash -c "./cockroach debug declarative-corpus-validate /tmp/corpus2039357818/corpus"
also from cockroach_debug_decl.log:
COMMAND_PROBLEM: exit status 134
(1) Node 1. Command with error:
| ```
| ./cockroach debug declarative-corpus-validate /tmp/corpus2039357818/corpus
| ```
| <truncated> ... 501b1
| runtime.goexit({})
| src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009b2fe8 sp=0xc0009b2fe0 pc=0x4d8d01
| created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
| cli.go:331 +0xe5
|
| goroutine 12 gp=0xc0022db340 m=nil [runnable]:
| runtime.gopark(0xc0009b37a8?, 0x2?, 0x0?, 0x0?, 0xc0009b371c?)
| GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009b35a8 sp=0xc0009b3588 pc=0x4a016e
| runtime.selectgo(0xc0009b37a8, 0xc0009b3718, 0x0?, 0x0, 0x0?, 0x1)
| GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009b36c8 sp=0xc0009b35a8 pc=0x4b20c5
| github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func2()
| cli.go:355 +0x9f fp=0xc0009b37e0 sp=0xc0009b36c8 pc=0x584fc3f
| runtime.goexit({})
| src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009b37e8 sp=0xc0009b37e0 pc=0x4d8d01
| created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
| cli.go:353 +0x1a5
| bash: line 8: 39888 Aborted (core dumped) bash -c "./cockroach debug declarative-corpus-validate /tmp/corpus2039357818/corpus"
Wraps: (2) COMMAND_PROBLEM
Wraps: (3) exit status 134
Error types: (1) *hintdetail.withDetail (2) errors.Cmd (3) *exec.ExitError
Dupe of https://github.com/cockroachdb/cockroach/issues/132523
roachtest.schemachange/mixed-versions-compat failed with artifacts on release-24.3.0-rc @ 49d902f212d10c8482298d777061ba153d5c16c9:
created by github.com/cockroachdb/cockroach/pkg/util/log.init.5 in goroutine 1
pkg/util/log/log_flush.go:79 +0x26
goroutine 19 gp=0xc001164700 m=nil [select, locked to thread]:
runtime.gopark(0xc0009b77a8?, 0x2?, 0x60?, 0x76?, 0xc0009b7794?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009b7638 sp=0xc0009b7618 pc=0x4a016e
runtime.selectgo(0xc0009b77a8, 0xc0009b7790, 0x0?, 0x0, 0x0?, 0x1)
GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009b7758 sp=0xc0009b7638 pc=0x4b20c5
runtime.ensureSigM.func1()
GOROOT/src/runtime/signal_unix.go:1034 +0x19f fp=0xc0009b77e0 sp=0xc0009b7758 pc=0x4cf05f
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009b77e8 sp=0xc0009b77e0 pc=0x4d8d01
created by runtime.ensureSigM in goroutine 12
GOROOT/src/runtime/signal_unix.go:1017 +0xc8
goroutine 13 gp=0xc0014af880 m=nil [chan receive]:
runtime.gopark(0xc000e00180?, 0x0?, 0x4?, 0x0?, 0xc0000deed0?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0000dee40 sp=0xc0000dee20 pc=0x4a016e
runtime.chanrecv(0xc000e00120, 0xc0000defb8, 0x1)
GOROOT/src/runtime/chan.go:583 +0x3bf fp=0xc0000deeb8 sp=0xc0000dee40 pc=0x468ddf
runtime.chanrecv1(0x8618a80?, 0xc001196000?)
GOROOT/src/runtime/chan.go:442 +0x12 fp=0xc0000deee0 sp=0xc0000deeb8 pc=0x4689f2
github.com/cockroachdb/cockroach/pkg/util/goschedstats.init.0.func1()
pkg/util/goschedstats/runnable.go:190 +0xe5 fp=0xc0000defe0 sp=0xc0000deee0 pc=0x262c4a5
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000defe8 sp=0xc0000defe0 pc=0x4d8d01
created by github.com/cockroachdb/cockroach/pkg/util/goschedstats.init.0 in goroutine 1
pkg/util/goschedstats/runnable.go:182 +0x1a
goroutine 16 gp=0xc0023e7880 m=nil [runnable]:
runtime.gopark(0xc0009b67a8?, 0x2?, 0x0?, 0x0?, 0xc0009b671c?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009b65a8 sp=0xc0009b6588 pc=0x4a016e
runtime.selectgo(0xc0009b67a8, 0xc0009b6718, 0x0?, 0x0, 0x0?, 0x1)
GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009b66c8 sp=0xc0009b65a8 pc=0x4b20c5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func2()
cli.go:355 +0x9f fp=0xc0009b67e0 sp=0xc0009b66c8 pc=0x5851c9f
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009b67e8 sp=0xc0009b67e0 pc=0x4d8d01
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
cli.go:353 +0x1a5
goroutine 38 gp=0xc000148000 m=nil [runnable]:
runtime.gopark(0xc0009b1fb0?, 0x2?, 0x0?, 0x0?, 0xc0009b1f8c?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009b1e38 sp=0xc0009b1e18 pc=0x4a016e
runtime.selectgo(0xc0009b1fb0, 0xc0009b1f88, 0x0?, 0x0, 0x0?, 0x1)
GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009b1f58 sp=0xc0009b1e38 pc=0x4b20c5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func1()
cli.go:333 +0x91 fp=0xc0009b1fe0 sp=0xc0009b1f58 pc=0x5852211
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009b1fe8 sp=0xc0009b1fe0 pc=0x4d8d01
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
cli.go:331 +0xe5
bash: line 8: 33696 Aborted (core dumped) bash -c "./cockroach debug declarative-corpus-validate /tmp/corpus513357535/corpus"
test artifacts and logs in: /artifacts/schemachange/mixed-versions-compat/run_1
Parameters:
ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
See: Grafana
- #133003 roachtest: schemachange/mixed-versions-compat failed [missing corpus file] [C-test-failure O-roachtest O-robot P-2 T-sql-foundations branch-release-24.3] - #132523 roachtest: schemachange/mixed-versions-compat failed [missing corpus file] [C-test-failure O-roachtest O-robot P-2 T-sql-foundations branch-master]
roachtest.schemachange/mixed-versions-compat failed with artifacts on release-24.3.0-rc @ 49d902f212d10c8482298d777061ba153d5c16c9:
created by runtime.ensureSigM in goroutine 20
GOROOT/src/runtime/signal_unix.go:1017 +0xc8
goroutine 12 gp=0xc000178a80 m=8 mp=0xc0013c2008 [syscall]:
runtime.notetsleepg(0xd391ea0, 0xffffffffffffffff)
GOROOT/src/runtime/lock_futex.go:246 +0x29 fp=0xc0000df7a0 sp=0xc0000df778 pc=0x46ee09
os/signal.signal_recv()
GOROOT/src/runtime/sigqueue.go:152 +0x29 fp=0xc0000df7c0 sp=0xc0000df7a0 pc=0x4d4e09
os/signal.loop()
GOROOT/src/os/signal/signal_unix.go:23 +0x13 fp=0xc0000df7e0 sp=0xc0000df7c0 pc=0xd90dd3
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000df7e8 sp=0xc0000df7e0 pc=0x4d8d01
created by os/signal.Notify.func1.1 in goroutine 20
GOROOT/src/os/signal/signal.go:151 +0x1f
goroutine 21 gp=0xc0010ffdc0 m=nil [chan receive]:
runtime.gopark(0xc0013821e0?, 0x0?, 0x4?, 0x0?, 0xc001305ed0?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc001305e40 sp=0xc001305e20 pc=0x4a016e
runtime.chanrecv(0xc001382180, 0xc001305fb8, 0x1)
GOROOT/src/runtime/chan.go:583 +0x3bf fp=0xc001305eb8 sp=0xc001305e40 pc=0x468ddf
runtime.chanrecv1(0x8618a80?, 0xc000140190?)
GOROOT/src/runtime/chan.go:442 +0x12 fp=0xc001305ee0 sp=0xc001305eb8 pc=0x4689f2
github.com/cockroachdb/cockroach/pkg/util/goschedstats.init.0.func1()
pkg/util/goschedstats/runnable.go:190 +0xe5 fp=0xc001305fe0 sp=0xc001305ee0 pc=0x262c4a5
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc001305fe8 sp=0xc001305fe0 pc=0x4d8d01
created by github.com/cockroachdb/cockroach/pkg/util/goschedstats.init.0 in goroutine 1
pkg/util/goschedstats/runnable.go:182 +0x1a
goroutine 25 gp=0xc0023b3a40 m=nil [runnable]:
runtime.gopark(0xc001302fa8?, 0x2?, 0x0?, 0x0?, 0xc001302f1c?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc001302da8 sp=0xc001302d88 pc=0x4a016e
runtime.selectgo(0xc001302fa8, 0xc001302f18, 0x0?, 0x0, 0x0?, 0x1)
GOROOT/src/runtime/select.go:327 +0x725 fp=0xc001302ec8 sp=0xc001302da8 pc=0x4b20c5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func2()
cli.go:355 +0x9f fp=0xc001302fe0 sp=0xc001302ec8 pc=0x5851c9f
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc001302fe8 sp=0xc001302fe0 pc=0x4d8d01
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
cli.go:353 +0x1a5
goroutine 13 gp=0xc001ef8000 m=nil [runnable]:
runtime.gopark(0xc0013087b0?, 0x2?, 0x0?, 0x0?, 0xc00130878c?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc001308638 sp=0xc001308618 pc=0x4a016e
runtime.selectgo(0xc0013087b0, 0xc001308788, 0xc0023cd7e0?, 0x0, 0xc0023b7668?, 0x1)
GOROOT/src/runtime/select.go:327 +0x725 fp=0xc001308758 sp=0xc001308638 pc=0x4b20c5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func1()
cli.go:333 +0x91 fp=0xc0013087e0 sp=0xc001308758 pc=0x5852211
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0013087e8 sp=0xc0013087e0 pc=0x4d8d01
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
cli.go:331 +0xe5
bash: line 8: 251776 Aborted (core dumped) bash -c "./cockroach debug declarative-corpus-validate /tmp/corpus3303233088/corpus"
test artifacts and logs in: /artifacts/schemachange/mixed-versions-compat/run_1
Parameters:
ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
See: Grafana
- #133003 roachtest: schemachange/mixed-versions-compat failed [missing corpus file] [C-test-failure O-roachtest O-robot P-2 T-sql-foundations branch-release-24.3] - #132523 roachtest: schemachange/mixed-versions-compat failed [invalid execution plan during ALTER TABLE ... ALTER COLUMN] [C-test-failure O-roachtest O-robot P-2 T-sql-foundations branch-master]
roachtest.schemachange/mixed-versions-compat failed with artifacts on release-24.3.0-rc @ dd28642dbab32aeaf42d647f0ed0a08780e8f8b0:
created by github.com/cockroachdb/cockroach/pkg/util/log.init.5 in goroutine 1
pkg/util/log/log_flush.go:79 +0x26
goroutine 50 gp=0xc00011a8c0 m=nil [select, locked to thread]:
runtime.gopark(0xc00100d7a8?, 0x2?, 0x60?, 0xd6?, 0xc00100d794?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc00100d638 sp=0xc00100d618 pc=0x4a016e
runtime.selectgo(0xc00100d7a8, 0xc00100d790, 0x0?, 0x0, 0x0?, 0x1)
GOROOT/src/runtime/select.go:327 +0x725 fp=0xc00100d758 sp=0xc00100d638 pc=0x4b20c5
runtime.ensureSigM.func1()
GOROOT/src/runtime/signal_unix.go:1034 +0x19f fp=0xc00100d7e0 sp=0xc00100d758 pc=0x4cf05f
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00100d7e8 sp=0xc00100d7e0 pc=0x4d8d01
created by runtime.ensureSigM in goroutine 36
GOROOT/src/runtime/signal_unix.go:1017 +0xc8
goroutine 51 gp=0xc000054380 m=7 mp=0xc00111a008 [syscall]:
runtime.notetsleepg(0xd391ea0, 0xffffffffffffffff)
GOROOT/src/runtime/lock_futex.go:246 +0x29 fp=0xc00100dfa0 sp=0xc00100df78 pc=0x46ee09
os/signal.signal_recv()
GOROOT/src/runtime/sigqueue.go:152 +0x29 fp=0xc00100dfc0 sp=0xc00100dfa0 pc=0x4d4e09
os/signal.loop()
GOROOT/src/os/signal/signal_unix.go:23 +0x13 fp=0xc00100dfe0 sp=0xc00100dfc0 pc=0xd90dd3
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00100dfe8 sp=0xc00100dfe0 pc=0x4d8d01
created by os/signal.Notify.func1.1 in goroutine 36
GOROOT/src/os/signal/signal.go:151 +0x1f
goroutine 37 gp=0xc00027ea80 m=nil [chan receive]:
runtime.gopark(0xc001118240?, 0x0?, 0x4?, 0x0?, 0xc0012d5ed0?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0012d5e40 sp=0xc0012d5e20 pc=0x4a016e
runtime.chanrecv(0xc001118180, 0xc0012d5fb8, 0x1)
GOROOT/src/runtime/chan.go:583 +0x3bf fp=0xc0012d5eb8 sp=0xc0012d5e40 pc=0x468ddf
runtime.chanrecv1(0x8618a80?, 0xc0000fc410?)
GOROOT/src/runtime/chan.go:442 +0x12 fp=0xc0012d5ee0 sp=0xc0012d5eb8 pc=0x4689f2
github.com/cockroachdb/cockroach/pkg/util/goschedstats.init.0.func1()
pkg/util/goschedstats/runnable.go:190 +0xe5 fp=0xc0012d5fe0 sp=0xc0012d5ee0 pc=0x262c4a5
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0012d5fe8 sp=0xc0012d5fe0 pc=0x4d8d01
created by github.com/cockroachdb/cockroach/pkg/util/goschedstats.init.0 in goroutine 1
pkg/util/goschedstats/runnable.go:182 +0x1a
goroutine 38 gp=0xc0022fc8c0 m=nil [runnable]:
runtime.gopark(0xc0012d47a8?, 0x2?, 0x0?, 0x0?, 0xc0012d471c?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0012d45a8 sp=0xc0012d4588 pc=0x4a016e
runtime.selectgo(0xc0012d47a8, 0xc0012d4718, 0x0?, 0x0, 0x0?, 0x1)
GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0012d46c8 sp=0xc0012d45a8 pc=0x4b20c5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func2()
cli.go:355 +0x9f fp=0xc0012d47e0 sp=0xc0012d46c8 pc=0x5851c9f
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0012d47e8 sp=0xc0012d47e0 pc=0x4d8d01
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
cli.go:353 +0x1a5
bash: line 8: 40893 Aborted (core dumped) bash -c "./cockroach debug declarative-corpus-validate /tmp/corpus181828279/corpus"
test artifacts and logs in: /artifacts/schemachange/mixed-versions-compat/run_1
Parameters:
ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
See: Grafana
- #135807 roachtest: schemachange/mixed-versions-compat failed [B-runtime-assertions-enabled C-test-failure O-roachtest O-robot T-sql-foundations branch-master release-blocker] - #133003 roachtest: schemachange/mixed-versions-compat failed [missing corpus file] [C-test-failure O-roachtest O-robot P-2 T-sql-foundations branch-release-24.3]
Note: This build has runtime assertions enabled. If the same failure was hit in a run without assertions enabled, there should be a similar failure without this message. If there isn't one, then this failure is likely due to an assertion violation or (assertion) timeout.
roachtest.schemachange/mixed-versions-compat failed with artifacts on release-24.3.0-rc @ ef2ebe96f3a26fd6d6716e2ec4a0c8c6fe0fb567:
created by runtime.ensureSigM in goroutine 24
GOROOT/src/runtime/signal_unix.go:1017 +0xc8
goroutine 51 gp=0xc0014ce1c0 m=5 mp=0xc000118008 [syscall]:
runtime.notetsleepg(0xcf878e0, 0xffffffffffffffff)
GOROOT/src/runtime/lock_futex.go:246 +0x29 fp=0xc0009b0fa0 sp=0xc0009b0f78 pc=0x46ee09
os/signal.signal_recv()
GOROOT/src/runtime/sigqueue.go:152 +0x29 fp=0xc0009b0fc0 sp=0xc0009b0fa0 pc=0x4d4e09
os/signal.loop()
GOROOT/src/os/signal/signal_unix.go:23 +0x13 fp=0xc0009b0fe0 sp=0xc0009b0fc0 pc=0xd91ff3
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009b0fe8 sp=0xc0009b0fe0 pc=0x4d8d01
created by os/signal.Notify.func1.1 in goroutine 24
GOROOT/src/os/signal/signal.go:151 +0x1f
goroutine 25 gp=0xc001288700 m=nil [chan receive]:
runtime.gopark(0x1?, 0x4?, 0xd0?, 0xce?, 0x2636eff?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc000fdce40 sp=0xc000fdce20 pc=0x4a016e
runtime.chanrecv(0xc00144a180, 0xc000fdcfb8, 0x1)
GOROOT/src/runtime/chan.go:583 +0x3bf fp=0xc000fdceb8 sp=0xc000fdce40 pc=0x468ddf
runtime.chanrecv1(0x8632820?, 0xc000252190?)
GOROOT/src/runtime/chan.go:442 +0x12 fp=0xc000fdcee0 sp=0xc000fdceb8 pc=0x4689f2
github.com/cockroachdb/cockroach/pkg/util/goschedstats.init.0.func1()
pkg/util/goschedstats/runnable.go:190 +0xe5 fp=0xc000fdcfe0 sp=0xc000fdcee0 pc=0x26371a5
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000fdcfe8 sp=0xc000fdcfe0 pc=0x4d8d01
created by github.com/cockroachdb/cockroach/pkg/util/goschedstats.init.0 in goroutine 1
pkg/util/goschedstats/runnable.go:182 +0x1a
goroutine 26 gp=0xc001fd6000 m=nil [runnable]:
runtime.gopark(0xc0009b47b0?, 0x2?, 0x0?, 0x0?, 0xc0009b478c?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009b4638 sp=0xc0009b4618 pc=0x4a016e
runtime.selectgo(0xc0009b47b0, 0xc0009b4788, 0x0?, 0x0, 0x0?, 0x1)
GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009b4758 sp=0xc0009b4638 pc=0x4b20c5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func1()
cli.go:333 +0x91 fp=0xc0009b47e0 sp=0xc0009b4758 pc=0x587fc11
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009b47e8 sp=0xc0009b47e0 pc=0x4d8d01
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
cli.go:331 +0xe5
goroutine 27 gp=0xc001fd61c0 m=nil [runnable]:
runtime.gopark(0xc0009b4fa8?, 0x2?, 0x0?, 0x0?, 0xc0009b4f1c?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009b4da8 sp=0xc0009b4d88 pc=0x4a016e
runtime.selectgo(0xc0009b4fa8, 0xc0009b4f18, 0x0?, 0x0, 0x0?, 0x1)
GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009b4ec8 sp=0xc0009b4da8 pc=0x4b20c5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func2()
cli.go:355 +0x9f fp=0xc0009b4fe0 sp=0xc0009b4ec8 pc=0x587f69f
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009b4fe8 sp=0xc0009b4fe0 pc=0x4d8d01
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
cli.go:353 +0x1a5
bash: line 8: 105969 Aborted (core dumped) bash -c "./cockroach debug declarative-corpus-validate /tmp/corpus3652212671/corpus"
test artifacts and logs in: /artifacts/schemachange/mixed-versions-compat/run_1
Parameters:
ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_runtimeAssertionsBuild=true
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
See: Grafana
- #135807 roachtest: schemachange/mixed-versions-compat failed [B-runtime-assertions-enabled C-test-failure O-roachtest O-robot T-sql-foundations branch-master release-blocker] - #133003 roachtest: schemachange/mixed-versions-compat failed [missing corpus file] [C-test-failure O-roachtest O-robot P-2 T-sql-foundations branch-release-24.3]
run_075539.754576284_n1_cockroach-debug-decl: 2024/11/21 07:55:39 cluster.go:2545: > ./cockroach debug declarative-corpus-validate /tmp/corpus3652212671/corpus
teamcity-17868557-1732171428-11-n1cpu4:[1]: ./cockroach debug declarati...
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 a panic has occurred!
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 +unexpected EOF
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 +(1) attached stack trace
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + -- stack trace:
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | github.com/cockroachdb/cockroach/pkg/cli.init.MaybeDecorateError.func68
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | pkg/cli/clierrorplus/decorate_error.go:67
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | github.com/spf13/cobra.(*Command).execute
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | external/com_github_spf13_cobra/command.go:916
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | github.com/spf13/cobra.(*Command).ExecuteC
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | external/com_github_spf13_cobra/command.go:1044
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | github.com/spf13/cobra.(*Command).Execute
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | external/com_github_spf13_cobra/command.go:968
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | github.com/cockroachdb/cockroach/pkg/cli.Run
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | cli.go:301
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | github.com/cockroachdb/cockroach/pkg/cli.doMain
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | cli.go:139
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | github.com/cockroachdb/cockroach/pkg/cli.Main
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | cli.go:64
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | main.main
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | pkg/cmd/cockroach-short/main.go:21
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | runtime.main
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | GOROOT/src/runtime/proc.go:271
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | runtime.goexit
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 + | src/runtime/asm_amd64.s:1695
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 +Wraps: (2) unexpected EOF
E241121 07:55:40.345441 1 1@util/log/logcrash/crash_reporting.go:192 [-] 1 +Error types: (1) *withstack.withStack (2) *errors.errorString
panic: unexpected EOF [recovered]
panic: unexpected EOF
goroutine 1 gp=0xc0000061c0 m=9 mp=0xc001380808 [running]:
panic({0x6230ce0?, 0xc710f20?})
GOROOT/src/runtime/panic.go:779 +0x158 fp=0xc00493d548 sp=0xc00493d498 pc=0x49c6f8
github.com/cockroachdb/cockroach/pkg/util/log/logcrash.RecoverAndReportPanic({0x867e300, 0xcf85280}, 0xc0027b4000)
pkg/util/log/logcrash/crash_reporting.go:135 +0x6e fp=0xc00493d598 sp=0xc00493d548 pc=0xf0d56e
github.com/cockroachdb/cockroach/pkg/cli.doMain.deferwrap1()
cli.go:137 +0x28 fp=0xc00493d5c0 sp=0xc00493d598 pc=0x587eb28
panic({0x6230ce0?, 0xc710f20?})
GOROOT/src/runtime/panic.go:770 +0x132 fp=0xc00493d670 sp=0xc00493d5c0 pc=0x49c6d2
github.com/cockroachdb/cockroach/pkg/cli.init.func17(0xc82d8e0, {0xc0028540f0?, 0x5127260?, 0xc00280cf00?})
declarative_corpus.go:38 +0x54c fp=0xc00493db18 sp=0xc00493d670 pc=0x586fb4c
github.com/cockroachdb/cockroach/pkg/cli.init.MaybeDecorateError.func68(0xc82d8e0?, {0xc0028540f0?, 0x1?, 0x1?})
pkg/cli/clierrorplus/decorate_error.go:67 +0x34 fp=0xc00493dc58 sp=0xc00493db18 pc=0x585c3d4
github.com/spf13/cobra.(*Command).execute(0xc82d8e0, {0xc0028540c0, 0x1, 0x1})
external/com_github_spf13_cobra/command.go:916 +0x882 fp=0xc00493dd78 sp=0xc00493dc58 pc=0x511d782
github.com/spf13/cobra.(*Command).ExecuteC(0xc827a00)
external/com_github_spf13_cobra/command.go:1044 +0x3a5 fp=0xc00493de30 sp=0xc00493dd78 pc=0x511dfc5
github.com/spf13/cobra.(*Command).Execute(...)
external/com_github_spf13_cobra/command.go:968
github.com/cockroachdb/cockroach/pkg/cli.Run(...)
cli.go:301
github.com/cockroachdb/cockroach/pkg/cli.doMain(0xc82d8e0, {0xc000dd5f2a, 0x21})
cli.go:139 +0x1e6 fp=0xc00493deb0 sp=0xc00493de30 pc=0x587ea46
github.com/cockroachdb/cockroach/pkg/cli.Main()
cli.go:64 +0x11f fp=0xc00493df40 sp=0xc00493deb0 pc=0x587e6ff
main.main()
pkg/cmd/cockroach-short/main.go:21 +0xf fp=0xc00493df50 sp=0xc00493df40 pc=0x5a53e0f
runtime.main()
GOROOT/src/runtime/proc.go:271 +0x29d fp=0xc00493dfe0 sp=0xc00493df50 pc=0x49fd1d
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00493dfe8 sp=0xc00493dfe0 pc=0x4d8d01
This is fixed by https://github.com/cockroachdb/cockroach/pull/135933 in master and 24.3, since this is low impact we are not fixing this in 24.3-rc and will close this once this branch gets closed.
roachtest.schemachange/mixed-versions-compat failed with artifacts on release-24.3.0-rc @ ef2ebe96f3a26fd6d6716e2ec4a0c8c6fe0fb567:
created by github.com/cockroachdb/cockroach/pkg/util/log.init.5 in goroutine 1
pkg/util/log/log_flush.go:79 +0x26
goroutine 37 gp=0xc00144e380 m=nil [select, locked to thread]:
runtime.gopark(0xc0000e57a8?, 0x2?, 0x60?, 0x56?, 0xc0000e5794?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0000e5638 sp=0xc0000e5618 pc=0x4a016e
runtime.selectgo(0xc0000e57a8, 0xc0000e5790, 0x0?, 0x0, 0x0?, 0x1)
GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0000e5758 sp=0xc0000e5638 pc=0x4b20c5
runtime.ensureSigM.func1()
GOROOT/src/runtime/signal_unix.go:1034 +0x19f fp=0xc0000e57e0 sp=0xc0000e5758 pc=0x4cf05f
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000e57e8 sp=0xc0000e57e0 pc=0x4d8d01
created by runtime.ensureSigM in goroutine 22
GOROOT/src/runtime/signal_unix.go:1017 +0xc8
goroutine 23 gp=0xc0010b41c0 m=nil [chan receive]:
runtime.gopark(0xc0000fa000?, 0x0?, 0x4?, 0x0?, 0xc00012bed0?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc00012be40 sp=0xc00012be20 pc=0x4a016e
runtime.chanrecv(0xc001010000, 0xc00012bfb8, 0x1)
GOROOT/src/runtime/chan.go:583 +0x3bf fp=0xc00012beb8 sp=0xc00012be40 pc=0x468ddf
runtime.chanrecv1(0x8618840?, 0xc001014000?)
GOROOT/src/runtime/chan.go:442 +0x12 fp=0xc00012bee0 sp=0xc00012beb8 pc=0x4689f2
github.com/cockroachdb/cockroach/pkg/util/goschedstats.init.0.func1()
pkg/util/goschedstats/runnable.go:190 +0xe5 fp=0xc00012bfe0 sp=0xc00012bee0 pc=0x262c4a5
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00012bfe8 sp=0xc00012bfe0 pc=0x4d8d01
created by github.com/cockroachdb/cockroach/pkg/util/goschedstats.init.0 in goroutine 1
pkg/util/goschedstats/runnable.go:182 +0x1a
goroutine 38 gp=0xc00201a700 m=nil [runnable]:
runtime.gopark(0xc00012f7b0?, 0x2?, 0x0?, 0x0?, 0xc00012f78c?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc00012f638 sp=0xc00012f618 pc=0x4a016e
runtime.selectgo(0xc00012f7b0, 0xc00012f788, 0x4?, 0x0, 0x0?, 0x1)
GOROOT/src/runtime/select.go:327 +0x725 fp=0xc00012f758 sp=0xc00012f638 pc=0x4b20c5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func1()
cli.go:333 +0x91 fp=0xc00012f7e0 sp=0xc00012f758 pc=0x58520b1
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc00012f7e8 sp=0xc00012f7e0 pc=0x4d8d01
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
cli.go:331 +0xe5
goroutine 39 gp=0xc0022b7500 m=nil [runnable]:
runtime.gopark(0xc00012ffa8?, 0x2?, 0x0?, 0x0?, 0xc00012ff1c?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0000f8da8 sp=0xc0000f8d88 pc=0x4a016e
runtime.selectgo(0xc0000f8fa8, 0xc00012ff18, 0x0?, 0x0, 0x0?, 0x1)
GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0000f8ec8 sp=0xc0000f8da8 pc=0x4b20c5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func2()
cli.go:355 +0x9f fp=0xc0000f8fe0 sp=0xc0000f8ec8 pc=0x5851b3f
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000f8fe8 sp=0xc0000f8fe0 pc=0x4d8d01
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
cli.go:353 +0x1a5
bash: line 8: 17976 Aborted (core dumped) bash -c "./cockroach debug declarative-corpus-validate /tmp/corpus3816497245/corpus"
test artifacts and logs in: /artifacts/schemachange/mixed-versions-compat/run_1
Parameters:
ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
See: Grafana
Note: This build has runtime assertions enabled. If the same failure was hit in a run without assertions enabled, there should be a similar failure without this message. If there isn't one, then this failure is likely due to an assertion violation or (assertion) timeout.
roachtest.schemachange/mixed-versions-compat failed with artifacts on release-24.3.0-rc @ 330209cefef4383a9d67ce7d86e2d3a9794ccd4f:
created by github.com/cockroachdb/cockroach/pkg/util/log.init.5 in goroutine 1
pkg/util/log/log_flush.go:79 +0x26
goroutine 36 gp=0xc001288540 m=nil [select, locked to thread]:
runtime.gopark(0xc0009b17a8?, 0x2?, 0x60?, 0x16?, 0xc0009b1794?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009b1638 sp=0xc0009b1618 pc=0x4a016e
runtime.selectgo(0xc0009b17a8, 0xc0009b1790, 0x0?, 0x0, 0x0?, 0x1)
GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009b1758 sp=0xc0009b1638 pc=0x4b20c5
runtime.ensureSigM.func1()
GOROOT/src/runtime/signal_unix.go:1034 +0x19f fp=0xc0009b17e0 sp=0xc0009b1758 pc=0x4cf05f
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009b17e8 sp=0xc0009b17e0 pc=0x4d8d01
created by runtime.ensureSigM in goroutine 10
GOROOT/src/runtime/signal_unix.go:1017 +0xc8
goroutine 50 gp=0xc0014c21c0 m=5 mp=0xc000118008 [syscall]:
runtime.notetsleepg(0xcf878e0, 0xffffffffffffffff)
GOROOT/src/runtime/lock_futex.go:246 +0x29 fp=0xc000f877a0 sp=0xc000f87778 pc=0x46ee09
os/signal.signal_recv()
GOROOT/src/runtime/sigqueue.go:152 +0x29 fp=0xc000f877c0 sp=0xc000f877a0 pc=0x4d4e09
os/signal.loop()
GOROOT/src/os/signal/signal_unix.go:23 +0x13 fp=0xc000f877e0 sp=0xc000f877c0 pc=0xd91ff3
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc000f877e8 sp=0xc000f877e0 pc=0x4d8d01
created by os/signal.Notify.func1.1 in goroutine 10
GOROOT/src/os/signal/signal.go:151 +0x1f
goroutine 11 gp=0xc001382700 m=nil [chan receive]:
runtime.gopark(0x0?, 0x4?, 0xd0?, 0xee?, 0x2636eff?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0000dee40 sp=0xc0000dee20 pc=0x4a016e
runtime.chanrecv(0xc0012e7c20, 0xc0000defb8, 0x1)
GOROOT/src/runtime/chan.go:583 +0x3bf fp=0xc0000deeb8 sp=0xc0000dee40 pc=0x468ddf
runtime.chanrecv1(0x8632820?, 0xc0002521e0?)
GOROOT/src/runtime/chan.go:442 +0x12 fp=0xc0000deee0 sp=0xc0000deeb8 pc=0x4689f2
github.com/cockroachdb/cockroach/pkg/util/goschedstats.init.0.func1()
pkg/util/goschedstats/runnable.go:190 +0xe5 fp=0xc0000defe0 sp=0xc0000deee0 pc=0x26371a5
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000defe8 sp=0xc0000defe0 pc=0x4d8d01
created by github.com/cockroachdb/cockroach/pkg/util/goschedstats.init.0 in goroutine 1
pkg/util/goschedstats/runnable.go:182 +0x1a
goroutine 12 gp=0xc00200f340 m=nil [runnable]:
runtime.gopark(0xc0009b4fa8?, 0x2?, 0x0?, 0x0?, 0xc0009b4f1c?)
GOROOT/src/runtime/proc.go:402 +0xce fp=0xc003907da8 sp=0xc003907d88 pc=0x4a016e
runtime.selectgo(0xc003907fa8, 0xc0009b4f18, 0x0?, 0x0, 0x0?, 0x1)
GOROOT/src/runtime/select.go:327 +0x725 fp=0xc003907ec8 sp=0xc003907da8 pc=0x4b20c5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func2()
cli.go:355 +0x9f fp=0xc003907fe0 sp=0xc003907ec8 pc=0x587f69f
runtime.goexit({})
src/runtime/asm_amd64.s:1695 +0x1 fp=0xc003907fe8 sp=0xc003907fe0 pc=0x4d8d01
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
cli.go:353 +0x1a5
bash: line 8: 25318 Aborted (core dumped) bash -c "./cockroach debug declarative-corpus-validate /tmp/corpus2322765729/corpus"
test artifacts and logs in: /artifacts/schemachange/mixed-versions-compat/run_1
Parameters:
ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_runtimeAssertionsBuild=true
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
See: Grafana
roachtest.schemachange/mixed-versions-compat failed with artifacts on release-24.3.0-rc @ 6d0a627cbefef712d05815b90b738e01805a618e:
Parameters:
ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
Help
See: roachtest README
See: How To Investigate (internal)
See: Grafana
Same failure on other branches
- #133003 roachtest: schemachange/mixed-versions-compat failed [missing corpus file] [C-test-failure O-roachtest O-robot P-2 T-sql-foundations branch-release-24.3] - #132523 roachtest: schemachange/mixed-versions-compat failed [missing corpus file] [C-test-failure O-roachtest O-robot P-2 T-sql-foundations branch-master]
/cc @cockroachdb/sql-foundations
This test on roachdash | Improve this report!
Jira issue: CRDB-44300