search

apache / incubator-xtable

Apache XTable (incubating) is a cross-table converter for lakehouse table formats that facilitates interoperability across data processing systems and query engines.
https://xtable.apache.org/
Apache License 2.0
848 stars 140 forks source link

Properly reflect rollbacks/restores in target tables #40

Open the-other-tim-brown opened 1 year ago

the-other-tim-brown commented 1 year ago

Right now when we see a rollback or restore in the source table, we just treat it as files being removed from the table. We should update this to instead issue a rollback command in the target tables so that the histories are more consistent between the source and target.

gzagarwal commented 6 months ago

What is the source and target in this scenario ? Actually i am trying to understand the code change so following the jira issues as well might ask silly question.

the-other-tim-brown commented 6 months ago

@gzagarwal The idea here is that the source can be any of the supported sources and target is any of the supported targets. The vision was that a rollback/restore to a previous point in time or commit would trigger the same in the target format if possible (fallback to current behavior of computing files to add/remove to the target format's view)