search

environmental-forecasting / model-ensembler

Model Ensemble tool for batch workflows on HPCs
https://pypi.org/project/model-ensembler/
MIT License
13 stars 0 forks source link

Register run directory removal to prevent future running #38

Open JimCircadian opened 2 years ago

JimCircadian commented 2 years ago

Now there are run results being stored the reporting is slightly misleading, all of the following jobs were previously processed. Copying/deleting the run should be flagged (optionally) to allow skips after the initial completion of the job, especially with repeating batches.

[08-04-22 11:58:44    :DEBUG   ] - Batch flop result #0 from run 0: job 4513298
[08-04-22 11:58:44    :DEBUG   ] - Batch flop result #1 from run 1: job 4513300
[08-04-22 11:58:44    :DEBUG   ] - Batch flop result #2 from run 2: job 4513299
[08-04-22 11:58:44    :DEBUG   ] - Batch flop result #3 from run 3: job None
[08-04-22 11:58:44    :WARNING ] - Result #3 for run 3 indicates unsuccessful submission, adding to indexes to skip
[08-04-22 11:58:44    :DEBUG   ] - Batch flop result #4 from run 4: job None
[08-04-22 11:58:44    :WARNING ] - Result #4 for run 4 indicates unsuccessful submission, adding to indexes to skip
[08-04-22 11:58:44    :DEBUG   ] - Batch flop result #5 from run 5: job 4513302
[08-04-22 11:58:44    :DEBUG   ] - Batch flop result #6 from run 6: job 4513297
[08-04-22 11:58:44    :DEBUG   ] - Batch flop result #7 from run 7: job None
[08-04-22 11:58:44    :WARNING ] - Result #7 for run 7 indicates unsuccessful submission, adding to indexes to skip
[08-04-22 11:58:44    :DEBUG   ] - Batch flop result #8 from run 8: job None
[08-04-22 11:58:44    :WARNING ] - Result #8 for run 8 indicates unsuccessful submission, adding to indexes to skip
[08-04-22 11:58:44    :DEBUG   ] - Batch flop result #9 from run 9: job None
[08-04-22 11:58:44    :WARNING ] - Result #9 for run 9 indicates unsuccessful submission, adding to indexes to skip
[08-04-22 11:58:44    :DEBUG   ] - Batch flop result #10 from run 10: job None
[08-04-22 11:58:44    :WARNING ] - Result #10 for run 10 indicates unsuccessful submission, adding to indexes to skip
[08-04-22 11:58:44    :DEBUG   ] - Batch flop result #11 from run 11: job None
[08-04-22 11:58:44    :WARNING ] - Result #11 for run 11 indicates unsuccessful submission, adding to indexes to skip