splitgraph / seafowl

Analytical database for data-driven Web applications 🪶
https://seafowl.io
Apache License 2.0
392 stars 9 forks source link

Datafusion 22 upgrade #360

Closed gruuya closed 1 year ago

gruuya commented 1 year ago

Upgrade Seafowl to DF 22 and Arrow 36, and thereby close #349.

Also refine some finesse(s) on UPDATE and DELETE partition pruning logic, and add corresponding tests. In particular, in case when we don't have any partitions to remove/add to the table state (because the WHERE clause completely misses all partitions), simply make a new table version and inherit all partitions as is. There are two alternatives that could be done here: