An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
This reverts commit b51b5b45595e46748339e42dbb0792e8b485a234.
disabling the cleanExpiredFiles API technically prevents removal manifests/manifest lists and users may not be running orphan file removal, so for those users manifests/manifest list may never be cleaned up. For now we can revert this patch to preserve the original behavior of just preventing data file removal so storage can be reclaimed via manifest/manifest list cleanup during uniform commits.
How was this patch tested?
Does this PR introduce any user-facing changes?
This reverts to cleaning up unreachable manifests/manifest lists during background commits, not really a direct user facing change.
Which Delta project/connector is this regarding?
Uniform
Description
This reverts commit b51b5b45595e46748339e42dbb0792e8b485a234. disabling the cleanExpiredFiles API technically prevents removal manifests/manifest lists and users may not be running orphan file removal, so for those users manifests/manifest list may never be cleaned up. For now we can revert this patch to preserve the original behavior of just preventing data file removal so storage can be reclaimed via manifest/manifest list cleanup during uniform commits.
How was this patch tested?
Does this PR introduce any user-facing changes?
This reverts to cleaning up unreachable manifests/manifest lists during background commits, not really a direct user facing change.