delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
https://delta.io
Apache License 2.0
7.62k stars 1.71k forks source link

Implementation for Lite Vacuum without parser changes #3757

Closed rajeshparangi closed 1 month ago

rajeshparangi commented 1 month ago

Which Delta project/connector is this regarding?

Description

This PR implements the logic of Lite Vacuum. Since the user surface is not defined yet, all of the code is behind a spark conf which is set to false by default.

How was this patch tested?

Modified existing tests to run in both Lite and full mode. Additionally, added new tests cases specific to Lite vacuum.

Does this PR introduce any user-facing changes?

NO