Currently cleanup time is set to 7 days - in cases where issues rise and many runs get stopped a lot of tables can accumulate. Setting a low value will result in deleting the table before it finished the run (in case of long running commands - such as ingesting a whole container)
A nice enhancement would be to periodically increase this value for extra hours. As long as the driver is running and havent finished processing the current DF it would increase the tables auto delete and retention policies.
Currently cleanup time is set to 7 days - in cases where issues rise and many runs get stopped a lot of tables can accumulate. Setting a low value will result in deleting the table before it finished the run (in case of long running commands - such as ingesting a whole container) A nice enhancement would be to periodically increase this value for extra hours. As long as the driver is running and havent finished processing the current DF it would increase the tables auto delete and retention policies.