roachtest: schemachange/mixed-versions-compat failed [missing corpus file]

cockroach-teamcity commented 3 days ago

roachtest.schemachange/mixed-versions-compat failed with artifacts on release-24.3 @ 4cbedefd790c75cb0f21f77ed8d917c8528a7d15:

runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000e07e8 sp=0xc0000e07e0 pc=0x4d8ce1
created by github.com/cockroachdb/cockroach/pkg/util/log.init.5 in goroutine 1
    github.com/cockroachdb/cockroach/pkg/util/log/log_flush.go:79 +0x26
goroutine 11 gp=0xc00011a540 m=nil [select, locked to thread]:
runtime.gopark(0xc0009b1fa8?, 0x2?, 0x60?, 0x1e?, 0xc0009b1f94?)
    GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009b1e38 sp=0xc0009b1e18 pc=0x4a014e
runtime.selectgo(0xc0009b1fa8, 0xc0009b1f90, 0x0?, 0x0, 0x0?, 0x1)
    GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009b1f58 sp=0xc0009b1e38 pc=0x4b20a5
runtime.ensureSigM.func1()
    GOROOT/src/runtime/signal_unix.go:1034 +0x19f fp=0xc0009b1fe0 sp=0xc0009b1f58 pc=0x4cf03f
runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009b1fe8 sp=0xc0009b1fe0 pc=0x4d8ce1
created by runtime.ensureSigM in goroutine 21
    GOROOT/src/runtime/signal_unix.go:1017 +0xc8
goroutine 37 gp=0xc000234700 m=4 mp=0xc0000e9808 [syscall]:
runtime.notetsleepg(0xd3866e0, 0xffffffffffffffff)
    GOROOT/src/runtime/lock_futex.go:246 +0x29 fp=0xc0009acfa0 sp=0xc0009acf78 pc=0x46ee09
os/signal.signal_recv()
    GOROOT/src/runtime/sigqueue.go:152 +0x29 fp=0xc0009acfc0 sp=0xc0009acfa0 pc=0x4d4de9
os/signal.loop()
    GOROOT/src/os/signal/signal_unix.go:23 +0x13 fp=0xc0009acfe0 sp=0xc0009acfc0 pc=0xd90893
runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009acfe8 sp=0xc0009acfe0 pc=0x4d8ce1
created by os/signal.Notify.func1.1 in goroutine 21
    GOROOT/src/os/signal/signal.go:151 +0x1f
goroutine 14 gp=0xc002313880 m=nil [runnable]:
runtime.gopark(0xc0009ac7b0?, 0x2?, 0x0?, 0x0?, 0xc0009ac78c?)
    GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009ac638 sp=0xc0009ac618 pc=0x4a014e
runtime.selectgo(0xc0009ac7b0, 0xc0009ac788, 0x4?, 0x0, 0x0?, 0x1)
    GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009ac758 sp=0xc0009ac638 pc=0x4b20a5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func1()
    github.com/cockroachdb/cockroach/pkg/cli/cli.go:333 +0x91 fp=0xc0009ac7e0 sp=0xc0009ac758 pc=0x5837ef1
runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009ac7e8 sp=0xc0009ac7e0 pc=0x4d8ce1
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
    github.com/cockroachdb/cockroach/pkg/cli/cli.go:331 +0xe5
goroutine 15 gp=0xc001f80540 m=nil [runnable]:
runtime.gopark(0xc0000e0fa8?, 0x2?, 0x0?, 0x0?, 0xc0000e0f1c?)
    GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0000e0da8 sp=0xc0000e0d88 pc=0x4a014e
runtime.selectgo(0xc0000e0fa8, 0xc0000e0f18, 0x0?, 0x0, 0x0?, 0x1)
    GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0000e0ec8 sp=0xc0000e0da8 pc=0x4b20a5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func2()
    github.com/cockroachdb/cockroach/pkg/cli/cli.go:355 +0x9f fp=0xc0000e0fe0 sp=0xc0000e0ec8 pc=0x583797f
runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000e0fe8 sp=0xc0000e0fe0 pc=0x4d8ce1
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
    github.com/cockroachdb/cockroach/pkg/cli/cli.go:353 +0x1a5
bash: line 8: 23981 Aborted                 (core dumped) bash -c "./cockroach debug declarative-corpus-validate /tmp/corpus3100893335/corpus"
test artifacts and logs in: /artifacts/schemachange/mixed-versions-compat/run_1

Parameters:

ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #132523 roachtest: schemachange/mixed-versions-compat failed [C-test-failure O-roachtest O-robot P-2 T-sql-foundations branch-master]

/cc @cockroachdb/sql-foundations

This test on roachdash | Improve this report!

Jira issue: CRDB-43400

cockroach-teamcity commented 2 days ago

Note: This build has runtime assertions enabled. If the same failure was hit in a run without assertions enabled, there should be a similar failure without this message. If there isn't one, then this failure is likely due to an assertion violation or (assertion) timeout.

roachtest.schemachange/mixed-versions-compat failed with artifacts on release-24.3 @ e466862ab36572bdc894cec84536b45472bdeb5b:

runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000defe8 sp=0xc0000defe0 pc=0x4d8ce1
created by github.com/cockroachdb/cockroach/pkg/util/log.init.5 in goroutine 1
    github.com/cockroachdb/cockroach/pkg/util/log/log_flush.go:79 +0x26
goroutine 50 gp=0xc0014f41c0 m=nil [select, locked to thread]:
runtime.gopark(0xc0009aafa8?, 0x2?, 0x60?, 0xae?, 0xc0009aaf94?)
    GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009aae38 sp=0xc0009aae18 pc=0x4a014e
runtime.selectgo(0xc0009aafa8, 0xc0009aaf90, 0x0?, 0x0, 0x0?, 0x1)
    GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009aaf58 sp=0xc0009aae38 pc=0x4b20a5
runtime.ensureSigM.func1()
    GOROOT/src/runtime/signal_unix.go:1034 +0x19f fp=0xc0009aafe0 sp=0xc0009aaf58 pc=0x4cf03f
runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009aafe8 sp=0xc0009aafe0 pc=0x4d8ce1
created by runtime.ensureSigM in goroutine 22
    GOROOT/src/runtime/signal_unix.go:1017 +0xc8
goroutine 51 gp=0xc0014f4380 m=4 mp=0xc0000e9808 [syscall]:
runtime.notetsleepg(0xcf75da0, 0xffffffffffffffff)
    GOROOT/src/runtime/lock_futex.go:246 +0x29 fp=0xc0009ab7a0 sp=0xc0009ab778 pc=0x46ee09
os/signal.signal_recv()
    GOROOT/src/runtime/sigqueue.go:152 +0x29 fp=0xc0009ab7c0 sp=0xc0009ab7a0 pc=0x4d4de9
os/signal.loop()
    GOROOT/src/os/signal/signal_unix.go:23 +0x13 fp=0xc0009ab7e0 sp=0xc0009ab7c0 pc=0xd91ab3
runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009ab7e8 sp=0xc0009ab7e0 pc=0x4d8ce1
created by os/signal.Notify.func1.1 in goroutine 22
    GOROOT/src/os/signal/signal.go:151 +0x1f
goroutine 35 gp=0xc001206a80 m=nil [runnable]:
runtime.gopark(0xc0009aefb0?, 0x2?, 0x0?, 0x0?, 0xc0009aef8c?)
    GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009aee38 sp=0xc0009aee18 pc=0x4a014e
runtime.selectgo(0xc0009aefb0, 0xc0009aef88, 0x0?, 0x0, 0x0?, 0x1)
    GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009aef58 sp=0xc0009aee38 pc=0x4b20a5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func1()
    github.com/cockroachdb/cockroach/pkg/cli/cli.go:333 +0x91 fp=0xc0009aefe0 sp=0xc0009aef58 pc=0x5864c11
runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009aefe8 sp=0xc0009aefe0 pc=0x4d8ce1
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
    github.com/cockroachdb/cockroach/pkg/cli/cli.go:331 +0xe5
goroutine 36 gp=0xc001206c40 m=nil [runnable]:
runtime.gopark(0xc0009af7a8?, 0x2?, 0x0?, 0x0?, 0xc0009af71c?)
    GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009af5a8 sp=0xc0009af588 pc=0x4a014e
runtime.selectgo(0xc0009af7a8, 0xc0009af718, 0x0?, 0x0, 0x0?, 0x1)
    GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009af6c8 sp=0xc0009af5a8 pc=0x4b20a5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func2()
    github.com/cockroachdb/cockroach/pkg/cli/cli.go:355 +0x9f fp=0xc0009af7e0 sp=0xc0009af6c8 pc=0x586469f
runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009af7e8 sp=0xc0009af7e0 pc=0x4d8ce1
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
    github.com/cockroachdb/cockroach/pkg/cli/cli.go:353 +0x1a5
bash: line 8: 16823 Aborted                 (core dumped) bash -c "./cockroach debug declarative-corpus-validate /tmp/corpus854622707/corpus"
test artifacts and logs in: /artifacts/schemachange/mixed-versions-compat/run_1

Parameters:

ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_fs=ext4
ROACHTEST_localSSD=true
ROACHTEST_runtimeAssertionsBuild=true
ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #132523 roachtest: schemachange/mixed-versions-compat failed [C-test-failure O-roachtest O-robot P-2 T-sql-foundations branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 1 day ago

roachtest.schemachange/mixed-versions-compat failed with artifacts on release-24.3 @ b2d2353b876af1748607e155ecdfed9d4bba29d3:

runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000e57e8 sp=0xc0000e57e0 pc=0x4d8ce1
created by github.com/cockroachdb/cockroach/pkg/util/log.init.5 in goroutine 1
    github.com/cockroachdb/cockroach/pkg/util/log/log_flush.go:79 +0x26
goroutine 7 gp=0xc001286a80 m=nil [select, locked to thread]:
runtime.gopark(0xc0009af7a8?, 0x2?, 0x60?, 0xf6?, 0xc0009af794?)
    GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009af638 sp=0xc0009af618 pc=0x4a014e
runtime.selectgo(0xc0009af7a8, 0xc0009af790, 0x0?, 0x0, 0x0?, 0x1)
    GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009af758 sp=0xc0009af638 pc=0x4b20a5
runtime.ensureSigM.func1()
    GOROOT/src/runtime/signal_unix.go:1034 +0x19f fp=0xc0009af7e0 sp=0xc0009af758 pc=0x4cf03f
runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009af7e8 sp=0xc0009af7e0 pc=0x4d8ce1
created by runtime.ensureSigM in goroutine 26
    GOROOT/src/runtime/signal_unix.go:1017 +0xc8
goroutine 8 gp=0xc001286c40 m=8 mp=0xc00131a008 [syscall]:
runtime.notetsleepg(0xd385720, 0xffffffffffffffff)
    GOROOT/src/runtime/lock_futex.go:246 +0x29 fp=0xc0009affa0 sp=0xc0009aff78 pc=0x46ee09
os/signal.signal_recv()
    GOROOT/src/runtime/sigqueue.go:152 +0x29 fp=0xc0009affc0 sp=0xc0009affa0 pc=0x4d4de9
os/signal.loop()
    GOROOT/src/os/signal/signal_unix.go:23 +0x13 fp=0xc0009affe0 sp=0xc0009affc0 pc=0xd90893
runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009affe8 sp=0xc0009affe0 pc=0x4d8ce1
created by os/signal.Notify.func1.1 in goroutine 26
    GOROOT/src/os/signal/signal.go:151 +0x1f
goroutine 51 gp=0xc0020cdc00 m=nil [runnable]:
runtime.gopark(0xc0009b2fb0?, 0x2?, 0x0?, 0x0?, 0xc0009b2f8c?)
    GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009b2e38 sp=0xc0009b2e18 pc=0x4a014e
runtime.selectgo(0xc0009b2fb0, 0xc0009b2f88, 0x0?, 0x0, 0x0?, 0x1)
    GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009b2f58 sp=0xc0009b2e38 pc=0x4b20a5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func1()
    github.com/cockroachdb/cockroach/pkg/cli/cli.go:333 +0x91 fp=0xc0009b2fe0 sp=0xc0009b2f58 pc=0x5837b31
runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009b2fe8 sp=0xc0009b2fe0 pc=0x4d8ce1
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
    github.com/cockroachdb/cockroach/pkg/cli/cli.go:331 +0xe5
goroutine 52 gp=0xc0020cddc0 m=nil [runnable]:
runtime.gopark(0xc0009b3fa8?, 0x2?, 0x0?, 0x0?, 0xc0009b3f1c?)
    GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009b3da8 sp=0xc0009b3d88 pc=0x4a014e
runtime.selectgo(0xc0009b3fa8, 0xc0009b3f18, 0x0?, 0x0, 0x0?, 0x1)
    GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009b3ec8 sp=0xc0009b3da8 pc=0x4b20a5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func2()
    github.com/cockroachdb/cockroach/pkg/cli/cli.go:355 +0x9f fp=0xc0009b3fe0 sp=0xc0009b3ec8 pc=0x58375bf
runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009b3fe8 sp=0xc0009b3fe0 pc=0x4d8ce1
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
    github.com/cockroachdb/cockroach/pkg/cli/cli.go:353 +0x1a5
bash: line 8: 17234 Aborted                 (core dumped) bash -c "./cockroach debug declarative-corpus-validate /tmp/corpus100725102/corpus"
test artifacts and logs in: /artifacts/schemachange/mixed-versions-compat/run_1

Parameters:

ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #132523 roachtest: schemachange/mixed-versions-compat failed [missing corpus file] [C-test-failure O-roachtest O-robot P-2 T-sql-foundations branch-master]

This test on roachdash | Improve this report!

cockroach-teamcity commented 7 hours ago

roachtest.schemachange/mixed-versions-compat failed with artifacts on release-24.3 @ 330f659a3f244d1386bb86c89a258d2da6063959:

created by github.com/cockroachdb/cockroach/pkg/util/log.init.5 in goroutine 1
    github.com/cockroachdb/cockroach/pkg/util/log/log_flush.go:78 +0x1a
goroutine 13 gp=0xc0011528c0 m=nil [chan receive]:
runtime.gopark(0xd90105?, 0x63f7540?, 0x1?, 0xa0?, 0xc0000def28?)
    GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0000dee98 sp=0xc0000dee78 pc=0x4a014e
runtime.chanrecv(0xc0013123c0, 0xc0000defb0, 0x1)
    GOROOT/src/runtime/chan.go:583 +0x3bf fp=0xc0000def10 sp=0xc0000dee98 pc=0x468ddf
runtime.chanrecv2(0xc0013123c0?, 0xc0000defc0?)
    GOROOT/src/runtime/chan.go:447 +0x12 fp=0xc0000def38 sp=0xc0000def10 pc=0x468a12
github.com/cockroachdb/cockroach/pkg/util/log.signalFlusher()
    github.com/cockroachdb/cockroach/pkg/util/log/log_flush.go:147 +0x89 fp=0xc0000defe0 sp=0xc0000def38 pc=0xdb3c69
runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0000defe8 sp=0xc0000defe0 pc=0x4d8ce1
created by github.com/cockroachdb/cockroach/pkg/util/log.init.5 in goroutine 1
    github.com/cockroachdb/cockroach/pkg/util/log/log_flush.go:79 +0x26
goroutine 20 gp=0xc0010be1c0 m=8 mp=0xc0012ec008 [syscall]:
runtime.notetsleepg(0xd385720, 0xffffffffffffffff)
    GOROOT/src/runtime/lock_futex.go:246 +0x29 fp=0xc0009aefa0 sp=0xc0009aef78 pc=0x46ee09
os/signal.signal_recv()
    GOROOT/src/runtime/sigqueue.go:152 +0x29 fp=0xc0009aefc0 sp=0xc0009aefa0 pc=0x4d4de9
os/signal.loop()
    GOROOT/src/os/signal/signal_unix.go:23 +0x13 fp=0xc0009aefe0 sp=0xc0009aefc0 pc=0xd90893
runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009aefe8 sp=0xc0009aefe0 pc=0x4d8ce1
created by os/signal.Notify.func1.1 in goroutine 13
    GOROOT/src/os/signal/signal.go:151 +0x1f
goroutine 22 gp=0xc000f1e000 m=nil [runnable]:
runtime.gopark(0xc0009b2fb0?, 0x2?, 0x0?, 0x0?, 0xc0009b2f8c?)
    GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009b2e38 sp=0xc0009b2e18 pc=0x4a014e
runtime.selectgo(0xc0009b2fb0, 0xc0009b2f88, 0x0?, 0x0, 0x0?, 0x1)
    GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009b2f58 sp=0xc0009b2e38 pc=0x4b20a5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func1()
    github.com/cockroachdb/cockroach/pkg/cli/cli.go:333 +0x91 fp=0xc0009b2fe0 sp=0xc0009b2f58 pc=0x5837e91
runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009b2fe8 sp=0xc0009b2fe0 pc=0x4d8ce1
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
    github.com/cockroachdb/cockroach/pkg/cli/cli.go:331 +0xe5
goroutine 51 gp=0xc0022bc380 m=nil [runnable]:
runtime.gopark(0xc0009adfa8?, 0x2?, 0x0?, 0x0?, 0xc0009adf1c?)
    GOROOT/src/runtime/proc.go:402 +0xce fp=0xc0009adda8 sp=0xc0009add88 pc=0x4a014e
runtime.selectgo(0xc0009adfa8, 0xc0009adf18, 0x0?, 0x0, 0x0?, 0x1)
    GOROOT/src/runtime/select.go:327 +0x725 fp=0xc0009adec8 sp=0xc0009adda8 pc=0x4b20a5
github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup.func2()
    github.com/cockroachdb/cockroach/pkg/cli/cli.go:355 +0x9f fp=0xc0009adfe0 sp=0xc0009adec8 pc=0x583791f
runtime.goexit({})
    src/runtime/asm_amd64.s:1695 +0x1 fp=0xc0009adfe8 sp=0xc0009adfe0 pc=0x4d8ce1
created by github.com/cockroachdb/cockroach/pkg/cli.debugSignalSetup in goroutine 1
    github.com/cockroachdb/cockroach/pkg/cli/cli.go:353 +0x1a5
bash: line 8: 40246 Aborted                 (core dumped) bash -c "./cockroach debug declarative-corpus-validate /tmp/corpus2468026157/corpus"
test artifacts and logs in: /artifacts/schemachange/mixed-versions-compat/run_1

Parameters:

ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_runtimeAssertionsBuild=false
ROACHTEST_ssd=0
Help

See: roachtest README

See: How To Investigate (internal)

See: Grafana

Same failure on other branches

- #132523 roachtest: schemachange/mixed-versions-compat failed [missing corpus file] [C-test-failure O-roachtest O-robot P-2 T-sql-foundations branch-master]

This test on roachdash | Improve this report!

spilchen commented 3 hours ago

@Dedej-Bergin this seems like the same as #132523. Assigning to you.

cockroachdb / cockroach

roachtest: schemachange/mixed-versions-compat failed [missing corpus file] #133003