search

apache / iceberg-rust

Apache Iceberg
https://rust.iceberg.apache.org/
Apache License 2.0
616 stars 138 forks source link

Add `ManifestEvaluator`, used to filter manifests in table scans #322

Closed sdd closed 5 months ago

sdd commented 6 months ago

This PR has been broken out of https://github.com/apache/iceberg-rust/pull/241 as that PR was getting too large.

It introduces ManfestEvaluator, which is used to apply the filter predicate from a table scan to a ManifestFile to see if it should be filtered out of the scan.

(Update: refactored based on changes to BoundPredicateVisitor recently merged to main)

sdd commented 5 months ago

@Fokko @liurenjie1024 @marvinlanhenke: I've rebased this on top of main now that the InclusiveProjection has been merged and it would be good to get some initial feedback.

sdd commented 5 months ago

There are intentionally a lot of todo!s in here. The aim is to get this PR merged so that more people can contribute implementations for the different visitor methods that need implementing, in subsequent PRs.

sdd commented 5 months ago

Thanks for the reviews, @marvinlanhenke and @liurenjie1024! All comments addressed and ready for re-review 😄

liurenjie1024 commented 5 months ago

Let's wait to see if others have comments.

liurenjie1024 commented 5 months ago

cc @Xuanwo @Fokko @marvinlanhenke PTAL

marvinlanhenke commented 5 months ago

@sdd Thanks a lot. LGTM

liurenjie1024 commented 5 months ago

There are intentionally a lot of todo!s in here. The aim is to get this PR merged so that more people can contribute implementations for the different visitor methods that need implementing, in subsequent PRs.

cc @Fokko I think @sdd is planning to add implementations in following smaller prs.