MaterializeInc / materialize

The Cloud Operational Data Store: use SQL to transform, deliver, and act on fast-changing data.
https://materialize.com
Other
5.72k stars 466 forks source link

persist: `EXPLAIN FILTER PUSHDOWN` improvements #27182

Open benesch opened 3 months ago

benesch commented 3 months ago

Follow up tasks from https://github.com/MaterializeInc/incidents-and-escalations/issues/24:

benesch commented 3 months ago

Copying in some context from https://github.com/MaterializeInc/materialize/pull/27091#issuecomment-2121623020:

This is a light abuse of the parameter, since it's only documented to apply to update-type statements currently

Nah, this seems great. We should just update the documentation! Tracked in #27182.

To fix that I think I need to mock this up as a "real" peek response, which is more work and may be too weird to merge?

Are you sure that's the way to go? You may just want to keep track of the task somewhere in struct Coordinator and add some bespoke cancellation handling in handle_privileged_cancel. There's some prior art in that function (e.g., subscribes are handled separately from peeks).

benesch commented 3 months ago

@bkirwi notes that the optimizer is currently pondering the same question (should it use stats that are potentially stale but immediately available), and it's quite possibly worth waiting for those questions to resolve before picking this work up again.