Currently, the output directory creation step occuers near the beginning of the process. importantly, prior to the pre-flight checks and data scraping, meaning that if any of those fail, the process will create an empty directory and then error out, leaving it empty.
Suggest moving output directory creation to later in the process, just prior to when we start writing files.
I could add on on.exit() hook to remove the directory if it doesn't contain any files (or on any condition), which should run even if there's an error.
Currently, the output directory creation step occuers near the beginning of the process. importantly, prior to the pre-flight checks and data scraping, meaning that if any of those fail, the process will create an empty directory and then error out, leaving it empty.
Suggest moving output directory creation to later in the process, just prior to when we start writing files.
https://github.com/RMI-PACTA/workflow.data.preparation/blob/819e1d1ab9144bb66fd2c000a93168340553f1bb/run_pacta_data_preparation.R#L84-L90
https://github.com/RMI-PACTA/workflow.data.preparation/blob/819e1d1ab9144bb66fd2c000a93168340553f1bb/run_pacta_data_preparation.R#L198-L224
AB#10858