Closed lordk911 closed 4 years ago
as the doc : https://iceberg.apache.org/spark-structured-streaming/
is waterdrop have any plan to support data lake component
@lordk911 你好,data lake有计划支持,只是目前没有用户给我们提过相关需求,没有需要落地的公司。如果你这边需要尽快落地,我们可以提高data lake支持的优先级
我们这边在做数据湖组件的技术选型吧,不过目前考虑到没有相应的ETL工具来支持,我看waterdrop是支持spark-structured-streaming的,应该能比较快的集成像iceberg这样的组件做sink吧。
as the doc : https://iceberg.apache.org/spark-structured-streaming/
is waterdrop have any plan to support data lake component