Support data files compaction

apache / iceberg-python

Apache PyIceberg

https://py.iceberg.apache.org/

Apache License 2.0

402 stars 147 forks source link

Open sungwy opened 1 month ago

sungwy commented 1 month ago

Introduce an API to compact data files. The first version of the API will do the following:

take a predicate expression as input parameter to find data files matching the filter that will be re-written
group data files by partitions and rewrite them using the same bin-packing constraints of the writer

sungwy commented 1 week ago

Unassigning to work on other near-term priorities