Closed seanmacb closed 6 months ago
To me, this seems like a problem in the parallelization
the strategy logs of the failed runs all are showing the mkdir error, which leads me to these lines in kn_strategy_sims_MI.py
If the directory already exists, then kn_strategy_sims_MI.py
will exit with the error code from above.
Why it is commonly notmoony that is making the directory and not encountering issues is unclear, especially when you consider that moony is being called first, so it should make the directory before the notmoony strategy does
If you add exist_ok = True
to the mkdir
call, it will not give you an error message, and should continue in strategy with no problem. I will implement that change now
Occasionally, notmoony or moony is not writing to .jsons
The issue is happening upstream, with the common error being
Traceback (most recent call last): File "/data/des70.a/data/desgw/O4/Main-Injector-O4b/python/knlc/kn_strategy_sims_MI.py", line 1809, in <module> kn_calc = KNCalc( File "/data/des70.a/data/desgw/O4/Main-Injector-O4b/python/knlc/kn_strategy_sims_MI.py", line 425, in __init__ os.mkdir(lowres_path) FileExistsError: [Errno 17] File exists: '/data/des70.a/data/desgw/O4/Main-Injector-O4b/OUTPUT/TESTING/MS240319t/PRELIMINARY_1/lowres'
or something of the sort