search

dbt-labs / metricflow

MetricFlow allows you to define, build, and maintain metrics in code.
https://docs.getdbt.com/docs/build/about-metricflow
Other
1.12k stars 92 forks source link

Track and propagate applied where filter specs to outer plan nodes #1302

Closed tlento closed 2 months ago

tlento commented 2 months ago

The PredicatePushdownOptimizer currently pushes predicates down along the DataflowPlan DAG from the outermost WhereConstraintNode to as close to the source node for that branch as possible. This results in duplicate where filter application, because the WhereConstraintNode does not have any way of evaluating whether or not a given set of where filter specs could be applied downstream.

This change adds the tracking mechanism for propagating the filters applied back up along the branch. As of now this is a tracking-change only - the selective application of these filters will follow shortly.

In addition to the added test cases for the propagation mechanism, the propagation mechanics were observed via testing several pushdown-enabled rendering tests with the log-cli-level=DEBUG flag set in pytest.

github-actions[bot] commented 2 months ago

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

tlento commented 2 months ago

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @tlento and the rest of your teammates on Graphite Graphite

tlento commented 2 months ago

Merge activity