Support iceberg table compaction

[rohankrao]

Starrocks supports writing into iceberg tables. It will be nice if starrocks supports iceberg table housekeeping.

Feature request

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Describe alternatives you've considered

Additional context

[alvin-celerdata]

@rohankrao Thanks for this suggestion, I wonder whether or not you have put starrocks over Iceberg in production?

[rohankrao]

No, not in production. This feature will help in production.

On Tue, 23 Jul, 2024, 2:45 am alvin, @.***> wrote:

@rohankrao https://github.com/rohankrao Thanks for this suggestion, I wonder whether or not you have put starrocks over Iceberg in production?

— Reply to this email directly, view it on GitHub https://github.com/StarRocks/starrocks/issues/48736#issuecomment-2243826958, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADXIE2BD3EUTCSTPZO4KMRTZNVY6TAVCNFSM6AAAAABLIX3GOWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBTHAZDMOJVHA . You are receiving this because you were mentioned.Message ID: @.***>

[Dshadowzh]

@rohankrao Which do you want, a fully managed compaction service or just a compaction interface you can integrated into your schedule system?

[rohankrao]

I will prefer a manual compaction command like you have for native tables. I am writing into iceberg from kafka externally and can trigger compaction when needed. Instead of using spark to do housekeeping, I want to use SR for that.

[nqvuong1998]

It would be beneficial if StarRocks supported Iceberg table maintenance features such as optimization, expiring snapshots, and removing orphan files.