Implement a utility tool for syncing parquet files to all three formats delta, hudi, iceberg using file listing

apache / incubator-xtable

Apache XTable (incubating) is a cross-table converter for lakehouse table formats that facilitates interoperability across data processing systems and query engines.

https://xtable.apache.org/

Apache License 2.0

868 stars 143 forks source link

Implement a utility tool for syncing parquet files to all three formats delta, hudi, iceberg using file listing #553

Open vinishjail97 opened 2 weeks ago

vinishjail97 commented 2 weeks ago

Feature Request / Improvement

Sub-task of main feature request https://github.com/apache/incubator-xtable/issues/550

Look at Using List files in the approach mentioned below. It's only a high level approach, PR author can change it and improve it as well. https://github.com/apache/incubator-xtable/issues/550#issuecomment-2386526797

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

sudharshanraja-db commented 1 week ago

Hi @vinishjail97 i am working on this task (last week was on and off with this) should be able to submit PR by next week just wanted to share the update here

vinishjail97 commented 1 week ago

okay, you can submit a draft PR as well if you are blocked on anything.

sudharshanraja-db commented 1 week ago

sure will do that