Hudi catalog improvement

StarRocks / starrocks

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries.

https://starrocks.io

Apache License 2.0

8.64k stars 1.74k forks source link

Hudi catalog improvement #46975

Open gohalo opened 2 months ago

gohalo commented 2 months ago

Enhancement

This issue is used to trace hudi related optimization, including metadata and sink.

[ ] refactor hudi table implement. [ ] refactor current remote file io implement. [ ] support incremental queries. [ ] maintain hudi catalog without hive. [ ] continuously optimize performance. [ ] support insert. [ ] support metadata table

alberttwong commented 1 month ago

I would add

Specifically support the following type of queries: COW Snapshot Queries COW Incremental Queries COW Incremental Queries (CDC) COW Bootstrap Queries MOR Snapshot Queries MOR Read-Optimized Queries MOR Incremental Queries MOR Incremental Queries (CDC) MOR Bootstrap queries(RO) MOR Bootstrap queries(snapshot) Time travel