nismod / open-gira

Open-data Global Infrastructure Risk/Resilience Analysis
https://nismod.github.io/open-gira/
MIT License
12 stars 3 forks source link

Results archival #97

Closed thomas-fred closed 1 year ago

thomas-fred commented 1 year ago

Here's a script for archiving open-gira results for transfer over the network, or just for safekeeping elsewhere on disk.

Invoking the script like so: ./archive_results.sh results ../open-gira_archive

Will create a compressed tarball from all the results (except the results/input directory). The archive structure is:

fred@coruisk:open-gira_archive$ tree 2022-08-25T151553+0100/
2022-08-25T151553+0100/
├── archive.tar.gz
└── README.md

With the README.md looking like:

open-gira repository state at time of archiving: 2022-08-25T151553+0100
N.B. Not necessarily the same state as when the results were created!

Commit: 98c36203d01bd61300f4f1bce935ca1016e557b7

Branch: 88_nightlight_validation

Status:
 M workflow/Snakefile
?? archive_results.sh
?? bundled_data/damage_curves/
?? config/dd.yaml
?? src/

Diff:
diff --git a/workflow/Snakefile b/workflow/Snakefile
index 7ea6bcc..07ca6cc 100644
--- a/workflow/Snakefile
+++ b/workflow/Snakefile
@@ -132,7 +132,8 @@ rule clean:
         rm -rf {config[output_dir]}/json &&
         rm -rf {config[output_dir]}/geoparquet &&
         rm -rf {config[output_dir]}/slices &&
-        rm -rf {config[output_dir]}/splits
+        rm -rf {config[output_dir]}/splits &&
+        rm -rf {config[output_dir]}/power_*
         """

 # Remove everything except the final results
@@ -143,5 +144,6 @@ rule clean_all:
         rm -rf {config[output_dir]}/json &&
         rm -rf {config[output_dir]}/geoparquet &&
         rm -rf {config[output_dir]}/slices &&
-        rm -rf {config[output_dir]}/splits
+        rm -rf {config[output_dir]}/splits &&
+        rm -rf {config[output_dir]}/power_*
         """