Closed def- closed 1 month ago
Just noticed, this happened after https://github.com/MaterializeInc/materialize/issues/28216, so maybe that is just the root cause.
This is also possibly fixed by https://github.com/MaterializeInc/materialize/pull/28324.🤞🏽
Seems so, haven't seen this fail with #28324 in https://buildkite.com/materialize/nightly/builds/8628
My bad, occurred again: https://ci-failures.dev.materialize.com/?key=test-failures&tfFilters=%5B%7B%22id%22%3A%22issue%22%2C%22value%22%3A%22materialize%2F28220%22%7D%5D Buildkite: https://buildkite.com/materialize/nightly/builds/8675 Code state included your PR
Forwarding a slack convo:
My current thinking is that this is caused by the non-existence of this line: https://github.com/MaterializeInc/materialize/blame/ac9a5555cefd9c024dc42098e9f164a32cf51b70/src/storage-client/src/storage_collections.rs#L2436.
We only taught StorageCollections to not do shard finalization in v0.108.x (https://github.com/MaterializeInc/materialize/pull/28118), and the test is trying to upgrade from v0.106.x to v0.107.x. I haven’t yet 100% confirmed this, because I didn’t yet figure out how to enable debug logs for a given crate in platform checks.
I have a draft PR for checking this theory: https://github.com/MaterializeInc/materialize/pull/28432
You were right, this hasn't occurred again after doing upgrades from X-3 instead of X-4. Since we're a week further, I'll switch us back to X-4 upgrades and close this if it doesn't reoccur.
What version of Materialize are you using?
15119a9478 (Pull Request #28191)
What is the issue?
Seen in https://buildkite.com/materialize/nightly/builds/8533#0190ab78-992b-482c-a715-b97127ac88b2:
ci-regexp: panicked at .* data should not be closed