informatics-lab / precip_rediagnosis

Project to use ML to re-diagnose precipitation fields from ensemble model fields
0 stars 0 forks source link

update data prep to autmotically cleanup after extract #81

Open stevehadd opened 1 year ago

stevehadd commented 1 year ago

currently the data extracted from mass is not generally cleaned up. The current workflow is to extract a temp dir on scratch, and then transfer to project/informatics_lab and the other directory is deleted. It would be good to specify a cleanup step as part of the data prep execution, and possibly also a transfer/upload step.