pndaproject / platform-data-mgmnt

PNDA components that manage aspects of datasets
Other
3 stars 20 forks source link

PNDA-4878: HDFS-cleaner fails to delete the last file that should have been deleted. #40

Closed arvind008 closed 6 years ago

arvind008 commented 6 years ago

Problem Statement:

PNDA-4878: HDFS-cleaner fails to delete the last file that should have been deleted.

Changes:

Modified the cleanup_on_size function to delete the last file that should be deleted based on size rule policy. Space consumed by HDFS directory should be updated after the conditional check for size threshold.

Test details

RHEL - PICO - HDP CENTOS - PICO - HDP