Closed cockroach-teamcity closed 1 month ago
roachtest.multitenant/distsql/instances=20/bundle=on/timeout=1 failed with artifacts on release-24.2 @ d6523e58ab3c09998535e88646ca16c1ead1ae4e:
(assertions.go:363).Fail:
Error Trace: github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_distsql.go:193
github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_distsql.go:47
main/pkg/cmd/roachtest/test_runner.go:1248
src/runtime/asm_amd64.s:1695
Error: Error message not equal:
expected: "pq: query execution canceled due to statement timeout"
actual : "dial tcp 34.139.74.251:29000: connect: connection refused"
Test: multitenant/distsql/instances=20/bundle=on/timeout=1
(require.go:177).EqualError: FailNow called
test artifacts and logs in: /artifacts/multitenant/distsql/instances=20/bundle=on/timeout=1/run_1
Parameters:
ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
See: Grafana
- #126313 roachtest: multitenant/distsql/instances=20/bundle=on/timeout=1 failed [C-test-failure O-roachtest O-robot T-sql-queries branch-master release-blocker]
roachtest.multitenant/distsql/instances=20/bundle=on/timeout=1 failed with artifacts on release-24.2 @ 23edce4c9826ebd4711b71c1029d2fc38a25f974:
(assertions.go:363).Fail:
Error Trace: github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_distsql.go:193
github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_distsql.go:47
main/pkg/cmd/roachtest/test_runner.go:1248
src/runtime/asm_amd64.s:1695
Error: Error message not equal:
expected: "pq: query execution canceled due to statement timeout"
actual : "dial tcp 35.237.225.5:29000: connect: connection refused"
Test: multitenant/distsql/instances=20/bundle=on/timeout=1
(require.go:177).EqualError: FailNow called
test artifacts and logs in: /artifacts/multitenant/distsql/instances=20/bundle=on/timeout=1/run_1
Parameters:
ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
See: Grafana
- #126313 roachtest: multitenant/distsql/instances=20/bundle=on/timeout=1 failed [C-test-failure O-roachtest O-robot T-sql-queries branch-master release-blocker]
roachtest.multitenant/distsql/instances=20/bundle=on/timeout=1 failed with artifacts on release-24.2 @ d4ef0343f38a35fb781b6343697b57a58131c68e:
(assertions.go:363).Fail:
Error Trace: github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_distsql.go:193
github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_distsql.go:47
main/pkg/cmd/roachtest/test_runner.go:1248
src/runtime/asm_amd64.s:1695
Error: Error message not equal:
expected: "pq: query execution canceled due to statement timeout"
actual : "dial tcp 35.231.21.22:29000: connect: connection refused"
Test: multitenant/distsql/instances=20/bundle=on/timeout=1
(require.go:177).EqualError: FailNow called
test artifacts and logs in: /artifacts/multitenant/distsql/instances=20/bundle=on/timeout=1/run_1
Parameters:
ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
See: Grafana
- #126313 roachtest: multitenant/distsql/instances=20/bundle=on/timeout=1 failed [C-test-failure O-roachtest O-robot T-sql-queries branch-master release-blocker]
roachtest.multitenant/distsql/instances=20/bundle=on/timeout=1 failed with artifacts on release-24.2 @ d4ef0343f38a35fb781b6343697b57a58131c68e:
(assertions.go:363).Fail:
Error Trace: github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_distsql.go:193
github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_distsql.go:47
main/pkg/cmd/roachtest/test_runner.go:1248
src/runtime/asm_amd64.s:1695
Error: Error message not equal:
expected: "pq: query execution canceled due to statement timeout"
actual : "dial tcp 34.148.16.48:29000: connect: connection refused"
Test: multitenant/distsql/instances=20/bundle=on/timeout=1
(require.go:177).EqualError: FailNow called
test artifacts and logs in: /artifacts/multitenant/distsql/instances=20/bundle=on/timeout=1/run_1
Parameters:
ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
See: roachtest README
See: How To Investigate (internal)
See: Grafana
- #126313 roachtest: multitenant/distsql/instances=20/bundle=on/timeout=1 failed [C-test-failure O-roachtest O-robot T-sql-queries branch-master release-blocker]
From this failure above:
* ERROR: a panic has occurred!
* use of Span after Finish. Span: optimizer. Finish previously called at: <stack not captured. Set debugUseAfterFinish>
* (1) attached stack trace
* -- stack trace:
* | runtime.gopanic
* | GOROOT/src/runtime/panic.go:770
* | [...repeated from below...]
* Wraps: (2) assertion failure
* Wraps: (3) attached stack trace
* -- stack trace:
* | github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish
* | github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182
* | github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).RecordingType
* | github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:442
* | github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).IsVerbose
* | github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:450
* | github.com/cockroachdb/cockroach/pkg/util/log.ExpensiveLogEnabled
* | github.com/cockroachdb/cockroach/pkg/util/log/log.go:64
* | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).sessionEventf
* | github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:4339
* | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).close
* | github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1247
* | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).runWithEx.func1
* | github.com/cockroachdb/cockroach/pkg/sql/internal.go:236
* | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).runWithEx.func2
* | github.com/cockroachdb/cockroach/pkg/sql/internal.go:257
* | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
* | github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:485
* | runtime.goexit
* | src/runtime/asm_amd64.s:1695
* Wraps: (4) use of Span after Finish. Span: optimizer. Finish previously called at: <stack not captured. Set debugUseAfterFinish>
* Error types: (1) *withstack.withStack (2) *assert.withAssertionFailure (3) *withstack.withStack (4) *errutil.leafError
*
*
* ERROR: a panic has occurred!
* use of Span after Finish. Span: prepare stmt. Finish previously called at: <stack not captured. Set debugUseAfterFinish>
* (1) attached stack trace
* -- stack trace:
* | runtime.gopanic
* | GOROOT/src/runtime/panic.go:770
* | [...repeated from below...]
* Wraps: (2) assertion failure
* Wraps: (3) attached stack trace
* -- stack trace:
* | github.com/cockroachdb/cockroach/pkg/util/tracing.(*Span).detectUseAfterFinish
* | github.com/cockroachdb/cockroach/pkg/util/tracing/span.go:182
* | github.com/cockroachdb/cockroach/pkg/util/tracing.WithParent
* | github.com/cockroachdb/cockroach/pkg/util/tracing/span_options.go:212
* | github.com/cockroachdb/cockroach/pkg/util/tracing.EnsureChildSpan
* | github.com/cockroachdb/cockroach/pkg/util/tracing/tracer.go:1627
* | github.com/cockroachdb/cockroach/pkg/sql.(*txnState).resetForNewSQLTxn
* | github.com/cockroachdb/cockroach/pkg/sql/txn_state.go:216
* | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).newConnExecutorWithTxn
* | github.com/cockroachdb/cockroach/pkg/sql/internal.go:461
* | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).initConnEx
* | github.com/cockroachdb/cockroach/pkg/sql/internal.go:343
* | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).runWithEx
* | github.com/cockroachdb/cockroach/pkg/sql/internal.go:226
* | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).execInternal
* | github.com/cockroachdb/cockroach/pkg/sql/internal.go:1269
* | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).QueryIteratorEx
* | github.com/cockroachdb/cockroach/pkg/sql/internal.go:886
* | github.com/cockroachdb/cockroach/pkg/sql/syntheticprivilegecache.(*Cache).readFromStorage
* | github.com/cockroachdb/cockroach/pkg/sql/syntheticprivilegecache/cache.go:133
* | github.com/cockroachdb/cockroach/pkg/sql/syntheticprivilegecache.(*Cache).Get.func1
* | github.com/cockroachdb/cockroach/pkg/sql/syntheticprivilegecache/cache.go:93
* | github.com/cockroachdb/cockroach/pkg/util/syncutil/singleflight.(*Group).doCall.func1
* | github.com/cockroachdb/cockroach/pkg/util/syncutil/singleflight/singleflight.go:389
* | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTask
* | github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:320
* | github.com/cockroachdb/cockroach/pkg/util/syncutil/singleflight.(*Group).doCall
* | github.com/cockroachdb/cockroach/pkg/util/syncutil/singleflight/singleflight.go:388
* | runtime.goexit
* | src/runtime/asm_amd64.s:1695
* Wraps: (4) use of Span after Finish. Span: prepare stmt. Finish previously called at: <stack not captured. Set debugUseAfterFinish>
* Error types: (1) *withstack.withStack (2) *assert.withAssertionFailure (3) *withstack.withStack (4) *errutil.leafError
*
Hm, I wonder whether the problem is this diff:
diff --git a/pkg/sql/syntheticprivilegecache/cache.go b/pkg/sql/syntheticprivilegecache/cache.go
index c098f6bc904..1ab1db3ed6f 100644
--- a/pkg/sql/syntheticprivilegecache/cache.go
+++ b/pkg/sql/syntheticprivilegecache/cache.go
@@ -90,7 +90,7 @@ func (c *Cache) Get(
}
privDesc, err := c.c.LoadValueOutsideOfCacheSingleFlight(ctx, fmt.Sprintf("%s-%d", spo.GetPath(), desc.GetVersion()),
func(loadCtx context.Context) (_ interface{}, retErr error) {
- return c.readFromStorage(ctx, txn, spo)
+ return c.readFromStorage(loadCtx, txn, spo)
})
if err != nil {
return nil, err
roachtest.multitenant/distsql/instances=20/bundle=on/timeout=1 failed with artifacts on release-24.2 @ 495d20e82f09b3436f1b00d7331146fe2042dd6a:
Parameters:
ROACHTEST_arch=amd64
ROACHTEST_cloud=gce
ROACHTEST_coverageBuild=false
ROACHTEST_cpu=4
ROACHTEST_encrypted=false
ROACHTEST_metamorphicBuild=false
ROACHTEST_ssd=0
Help
See: roachtest README
See: How To Investigate (internal)
See: Grafana
Same failure on other branches
- #126313 roachtest: multitenant/distsql/instances=20/bundle=on/timeout=1 failed [C-test-failure O-roachtest O-robot T-sql-queries branch-master release-blocker]
This test on roachdash | Improve this report!
Jira issue: CRDB-40216