search

RMI-PACTA / workflow.data.preparation

The goal of `workflow.data.preparation` is to prepare all of the necessary data inputs for the Transition Monitor web application.
Other
2 stars 0 forks source link

Move output directory creation to after pre-flight activities #182

Open AlexAxthelm opened 8 months ago

AlexAxthelm commented 8 months ago

Currently, the output directory creation step occuers near the beginning of the process. importantly, prior to the pre-flight checks and data scraping, meaning that if any of those fail, the process will create an empty directory and then error out, leaving it empty.

Suggest moving output directory creation to later in the process, just prior to when we start writing files.

https://github.com/RMI-PACTA/workflow.data.preparation/blob/819e1d1ab9144bb66fd2c000a93168340553f1bb/run_pacta_data_preparation.R#L84-L90

https://github.com/RMI-PACTA/workflow.data.preparation/blob/819e1d1ab9144bb66fd2c000a93168340553f1bb/run_pacta_data_preparation.R#L198-L224

AB#10858

cjyetman commented 8 months ago

I could add on on.exit() hook to remove the directory if it doesn't contain any files (or on any condition), which should run even if there's an error.

AlexAxthelm commented 8 months ago

hmm. that might be a simple solution.