Eventual-Inc / Daft

Distributed data engine for Python/SQL designed for the cloud, powered by Rust
https://getdaft.io
Apache License 2.0
2.34k stars 164 forks source link

Allow using `UnityCatalogTable` in `DataFrame.write_deltalake` #3336

Open kevinzwang opened 2 days ago

kevinzwang commented 2 days ago

Is your feature request related to a problem?

You can create a Unity Catalog table in Daft using daft.unity_catalog.UnityCatalog.load_table(tbl). At the moment, you can only use that table with daft.read_deltalake.

Describe the solution you'd like

We should also be able to similarly use it for DataFrame.write_deltalake.

Describe alternatives you've considered

We can extract the table URI and io config from the unity catalog table and pass those into write_deltalake manually, but that is not preferred.

Additional Context

No response

Would you like to implement a fix?

No