Before expiring snapshots is there need to provide history snapshot file statistics - Githubissues

apache / iceberg

Apache Iceberg

https://iceberg.apache.org/

Apache License 2.0

6.46k stars 2.23k forks source link

Before expiring snapshots is there need to provide history snapshot file statistics #11213

Open MichaelDeSteven opened 1 month ago

MichaelDeSteven commented 1 month ago

Feature Request / Improvement

context: when I want to keep the size of table small by expiring snapshots and delete orphan files, I find that there is no any way to estimate the files that can be deleted after the snapshot expiration before it occurs
suggest: Perhaps we can expose the total size of the files to be deleted after snapshot expiration through SQL or procedure.

Query engine

Spark

Willingness to contribute

[X] I can contribute this improvement/feature independently
[X] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
[ ] I cannot contribute this improvement/feature at this time

manuzhang commented 1 month ago

It might be valuable to add a dry_run option to expire_snapshots like remove_orphan_files.

MichaelDeSteven commented 1 month ago

It might be valuable to add a dry_run option to expire_snapshots like remove_orphan_files.

good idea! When dry_run is true, don't actually remove files but return info.