Open edavisau opened 11 months ago
Note that a workaround for now is to pushdown manually until the filter pushdown is implemented in polars
The simplest way being
pl.scan_pyarrow_dataset(
dataset
.filter(ds.field("date_col") == base_datetime)
)
.filter(pl.col("int_col").gt(2))
This only applies to filter expressions which can be expressed in pyarrow.
Checks
[X] I have checked that this issue has not already been reported.
[X] I have confirmed this bug exists on the latest version of Polars.
Reproducible example
Log output
No response
Issue description
If you take a filter which is supported by the pyarrow pushdown, and then add another filter which is not supported, the first filter is no longer pushed down.
(Was not sure whether this should be classified as a bug or enhancement)
Expected behavior
In the second example above, we should see something like
Installed versions