search

dask-contrib / dask-deltatable

A Delta Lake reader for Dask
BSD 3-Clause "New" or "Revised" License
44 stars 14 forks source link

What are the limitations of to_deltalake? #63

Open selvavm opened 9 months ago

selvavm commented 9 months ago

First of all, thanks for the effort.

I am evaluating Dask for my scientific computation which reads delta table and writes results as delta table. The readme says there is a limitation while writing but unclear on what it is. Can you please share some details?

Note - I also saw this page where they explain using Pandas but unsure if I can use write_deltalake in Dask.

jacobtomlinson commented 8 months ago

@fjetter @jrbourbeau can either of you comment on what the limitations are around writing to delta lake?