An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
When dropping a feature, DeltaLog now checks both backfilled and unbackfilled deltas for any traces of the feature before confirming it's safe to drop.
However, feature dropping currently does a checkpoint before detecting feature traces in the history, so there are no unbackfilled deltas at that point.
Which Delta project/connector is this regarding?
Description
When dropping a feature, DeltaLog now checks both backfilled and unbackfilled deltas for any traces of the feature before confirming it's safe to drop.
However, feature dropping currently does a checkpoint before detecting feature traces in the history, so there are no unbackfilled deltas at that point.
How was this patch tested?
UTs
Does this PR introduce any user-facing changes?
No