Closed Dandandan closed 1 month ago
I believe this is a duplicate of https://github.com/apache/datafusion/issues/4028, which @itsjunetime completed in https://github.com/apache/datafusion/pull/12135
Tests are here https://github.com/apache/datafusion/blob/main/datafusion/sqllogictest/test_files/parquet_filter_pushdown.slt
Let me know if I got that incorrect
Is your feature request related to a problem or challenge?
When
pushdown_filters
is enabled, DF should be able to eliminate the subsequent filter. When enabling the option for tpc-h benchmark, theFilterExec
andl_shipdate
projection is still present in the plans.For example query 3 we can see the filter:
Describe the solution you'd like
Remove the filter when.
We probably need to make some changes to
TableProvider
FileFormat
to support removing the filter based on the file format.Describe alternatives you've considered
No response
Additional context
No response