[Feature]: Support integration with Apache Hudi

[zyclove]

We have been using hudi as a data lake. Looking forward to supporting.

[zhoujinsong]

Hi, @zyclove

Thanks for your feed back. Supporting hudi is a very useful feature for arctic and we are planning put it into our roadmap. But It has a lot work to do in order to achieve this goal. And we are very pleased to welcome you to join the discussing and designing for this feature.

[melin]

Hi, @zyclove

Thanks for your feed back. Supporting hudi is a very useful feature for arctic and we are planning put it into our roadmap. But It has a lot work to do in order to achieve this goal. And we are very pleased to welcome you to join the discussing and designing for this feature.

目前比较难的是，hudi 没有想iceberg 保留catalog 扩展能力，社区还在讨论中，需要等很久

[zyclove]

请问现在社区有进度吗？很希望可以列出方案和计划，一起共同搞起来。现在很多特性确实hudi支持的很不错，hudi线上使用公司也特别多，对Arctic这种元数据管理服务依赖也很强烈。能不能大佬们讨论讨论搞个计划呢？

Hudi vs Delta Lake vs Iceberg: https://www.onehouse.ai/blog/apache-hudi-vs-delta-lake-vs-apache-iceberg-lakehouse-feature-comparison

hudi很多特性我们一直在线上使用，很期待可以支持一下哦。 @zhoujinsong @melin @fantasyni @radiumce

[zhoujinsong]

@zyclove Thanks a lot for bringing this feature up again! I must admit that right now the Arctic community has no clear plan for Hudi's integration. However, I think we can start discussing what value the Arctic can bring up to Hudi users after integration so that we can develop a more detailed integration plan later.

As far as I can see Arctic can bring the following values to Hudi users after integration：

• Centered optimizing task scheduling for Hudi tables to improve resource usage and stability of table optimizing tasks（compaction、clustering、cleaning）
• A web-based dashboard to show table information and metrics.

However, I would like to get more input from Hudi users about this question, so I would also like to hear your opinion.

[zyclove]

目前hudi社区也已经有元数据管理服务，也提供接口，现在是不是对接管理开发也更容易了，能不能加快一下排期呢？

[shidayang]

目前hudi社区也已经有元数据管理服务，也提供接口，现在是不是对接管理开发也更容易了，能不能加快一下排期呢？

We are very interested in integrating Hudi. Are you interested in driving this feature?

[baiyangtx]

目前hudi社区也已经有元数据管理服务，也提供接口，现在是不是对接管理开发也更容易了，能不能加快一下排期呢？

As far as I know, Hudi has its own Compaction service. What additional capabilities do you expect Amoro to provide for Hudi?

Do you want visualized Compaction management?

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.