[BUG][Spark] Logs are not compacted

[Minashraf]

Bug

Which Delta project/connector is this regarding?

• [X] Spark
• [ ] Standalone
• [ ] Flink
• [ ] Kernel
• [ ] Other (fill in here)

Describe the problem

Logs don't seem to be deleted after retention or after checkpoint

Steps to reproduce

I am using zeppelin notebook and loading my data from HDFS do an update I ran this command many times and I have a lot of checkpoints on hdfs but none of the logs are deleted Here is my table description

Observed results

None of the logs are deleted +100 files

Expected results

older logs to be deleted especially the ones before the checkpoint

Further details

Environment information

• Delta Lake version: delta-core_2.12-2.0.2.jar
• Spark version: 3.2.1
• Scala version:

Willingness to contribute

The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?

• [ ] Yes. I can contribute a fix for this bug independently.
• [X] Yes. I would be willing to contribute a fix for this bug with guidance from the Delta Lake community.
• [ ] No. I cannot contribute a bug fix at this time.

[Minashraf]

Found the solution minimum retention is 1 day and can't be converted to hours or minutes