search

datafuselabs / databend

𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com
https://docs.databend.com
Other
7.41k stars 713 forks source link

feat: Integrate with delta lake #7974

Open Xuanwo opened 1 year ago

Xuanwo commented 1 year ago

Summary

Delta lake is an open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs.

Allow databend read data from delta lake seems interesting. We can use delta-rs.

teckick commented 1 year ago

Does it mean using Databend as a compute engine for Delta, just like Spark / Presto?

Xuanwo commented 1 year ago

Does it mean using Databend as a compute engine for Delta, just like Spark / Presto?

Yes, make databend-query can query over delta lake directly. Integrate delta-rs is not suitable for us now, we need to port delta-rs to opendal first.

davidgasquez commented 1 year ago

Hey there! Are you planning on working on this one in 2023? Didn't saw Delta @ https://github.com/datafuselabs/databend/issues/9448. :cry:

sundy-li commented 1 year ago

Hey there! Are you planning on working on this one in 2023? Didn't saw Delta @ #9448. 😢

Thanks, it's added!