[better error handling] buildstock could handle duplicated files better

[wenyikuang]

If we run buildstock_local for same yml file twice. It will raise the FileExistsError .

DEBUG:2023-12-29 05:42:52:buildstockbatch.local:Compressing simulation outputs to /App2/baseline_test_output/simulation_output/simulations_job0.tar.gz
Traceback (most recent call last):
  File "/usr/local/bin/buildstock_local", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/site-packages/buildstockbatch/utils.py", line 120, in run_with_error_capture
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/buildstockbatch/local.py", line 471, in main
    batch.process_results()
  File "/usr/local/lib/python3.8/site-packages/buildstockbatch/base.py", line 911, in process_results
    postprocessing.combine_results(fs, self.results_dir, self.cfg, do_timeseries=do_timeseries)
  File "/usr/local/lib/python3.8/site-packages/buildstockbatch/postprocessing.py", line 405, in combine_results
    fs.makedirs(dr)
  File "/usr/local/lib/python3.8/site-packages/fsspec/implementations/local.py", line 54, in makedirs
    os.makedirs(path, exist_ok=exist_ok)
  File "/usr/local/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/App2/baseline_test_output/results_csvs'
root@f113d48994f9:/App2# rm -rf /App2/baseline_test_output/
root@f113d48994f9:/App2# buildstock_local ymls/bsb-integration-test-baseline.yml

I will fix it later.

[nmerket]

Ok, yeah, I thought we had it throw an error on the project validation step if the output directory exists. There's some nuance to it because you may want to run postprocessing on a run that's already done. Ultimately, we do want to have some sort of error if you're trying to overwrite existing results.

[wenyikuang]

Ok, make sense. Thanks for elaboration. Close it.