The last cache TableProvider has code to handle predicate pushdown here, however, there are no tests to verify that it is actually working as intended.
https://github.com/influxdata/influxdb/pull/25566 used a custom ExecutionPlan to verify the predicate pushdown is working, so we may consider following that example, since the code in the metadata cache is very similar to the last cache.
The idea is to capture the predicates evaluated so they can be displayed in the EXPLAIN output of a query that contains predicates destined for the cache.
This has the added benefit of validating that the pushdown is working, but also gives us the means to diagnose the same issue should we suspect pushdown is not working in the wild.
Alternatives
We could write lower level tests that validate the conversion function directly, but this would not have the added benefit mentioned above of also providing external visibility via EXPLAIN.
Should the pushdown not be working, we may want to adopt a similar strategy to the metadata cache to converting filter Exprs using DataFusion's LiteralGuarantee
Problem
The last cache
TableProviderhas code to handle predicate pushdown here, however, there are no tests to verify that it is actually working as intended.So, in light of https://github.com/influxdata/influxdb/issues/25550 and https://github.com/influxdata/influxdb/issues/25550, we should have tests in place to verify that the predicate pushdown works.
Proposed solution
https://github.com/influxdata/influxdb/pull/25566 used a custom
ExecutionPlanto verify the predicate pushdown is working, so we may consider following that example, since the code in the metadata cache is very similar to the last cache.The idea is to capture the predicates evaluated so they can be displayed in the
EXPLAINoutput of a query that contains predicates destined for the cache.This has the added benefit of validating that the pushdown is working, but also gives us the means to diagnose the same issue should we suspect pushdown is not working in the wild.
Alternatives
We could write lower level tests that validate the conversion function directly, but this would not have the added benefit mentioned above of also providing external visibility via
EXPLAIN.Additional context
Here is the custom implementor of the
ExecutionPlantrait used by the metadata cache: https://github.com/influxdata/influxdb/blob/20c74a7987422aeaf89bf24b2707ce1e5acc08bd/influxdb3_cache/src/meta_cache/table_function.rs#L251-L276Should the pushdown not be working, we may want to adopt a similar strategy to the metadata cache to converting filter
Exprs using DataFusion'sLiteralGuarantee