apache / amoro

Apache Amoro (incubating) is a Lakehouse management system built on open data lake formats.
https://amoro.apache.org/
Apache License 2.0
849 stars 278 forks source link

[AMORO-2861]: Support display Hudi table metadata in Amoro Dashboard #2877

Closed baiyangtx closed 1 month ago

baiyangtx commented 4 months ago

Why are the changes needed?

Close #2861.

Brief change log

How was this patch tested?

Documentation

baiyangtx commented 3 months ago

Support choose Hudi format when create hive/hadoop catalog.

image

Support display Hudi tables in catalog, with hudi icon.

image

Table details

Support display Hudi table details, include table summary, scheme, metrics, table type and properties image

Support display partitions and files in partitions.

Due to limit of hudi api, some field without values. image image

Support display timeline as snapshots

image image Due to limit of hudi api, records statistic is missed, and file count is incorrect for deltacommit.

Support display compact/cluster instant as optimizing process

image image

Only show completed instant, and show compact as minor optimize, cluster as major optimize.

DDL

Due to limit of hudi api, DDL history is empty.

zhoujinsong commented 3 months ago

Some check style errors exist, which you may want to fix. @baiyangtx

czy006 commented 1 month ago

cc @majin1102 @zhoujinsong