SSantosLab / Main-Injector

2 stars 4 forks source link

Add moony vs. notmoony handling #37

Closed seanmacb closed 6 months ago

seanmacb commented 6 months ago

Occasionally, notmoony or moony is not writing to .jsons

The issue is happening upstream, with the common error being

Traceback (most recent call last): File "/data/des70.a/data/desgw/O4/Main-Injector-O4b/python/knlc/kn_strategy_sims_MI.py", line 1809, in <module> kn_calc = KNCalc( File "/data/des70.a/data/desgw/O4/Main-Injector-O4b/python/knlc/kn_strategy_sims_MI.py", line 425, in __init__ os.mkdir(lowres_path) FileExistsError: [Errno 17] File exists: '/data/des70.a/data/desgw/O4/Main-Injector-O4b/OUTPUT/TESTING/MS240319t/PRELIMINARY_1/lowres'

or something of the sort

seanmacb commented 6 months ago

To me, this seems like a problem in the parallelization

the strategy logs of the failed runs all are showing the mkdir error, which leads me to these lines in kn_strategy_sims_MI.py

If the directory already exists, then kn_strategy_sims_MI.py will exit with the error code from above.

Why it is commonly notmoony that is making the directory and not encountering issues is unclear, especially when you consider that moony is being called first, so it should make the directory before the notmoony strategy does

seanmacb commented 6 months ago

If you add exist_ok = True to the mkdir call, it will not give you an error message, and should continue in strategy with no problem. I will implement that change now