search

delta-io / delta-rs

A native Rust library for Delta Lake, with bindings into Python
https://delta-io.github.io/delta-rs/
Apache License 2.0
1.97k stars 365 forks source link

fix: enable parquet pushdown for DeltaScan via TableProvider impl for DeltaTable #2618

Open alexwilcoxson-rel opened 5 days ago

alexwilcoxson-rel commented 5 days ago

Description

The DeltaScanBuilder used by DeltaTable's TableProvider impl does not specify DeltaScanConfig. The builder uses Default for DeltaScanConfig and thus enable_parquet_pushdown is disabled.

This change makes it so DeltaScanConfig is an Option on the scan builder. If unset during build a default value will be created using the DeltaScanConfigBuilder rather than DeltaScanConfig::Default. The former has enable_parquet_pushdown defaulted to true. I considered changing the latter but it is used in a couple other places where I was not sure of the impact.

Related Issue(s)

Documentation

github-actions[bot] commented 5 days ago

ACTION NEEDED

delta-rs follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.