search

cxcsds / ciao-contrib

Extra scripts and code to enhance the capabilities of CIAO.
GNU General Public License v3.0
8 stars 6 forks source link

fall back on 'spawn' if 'fork' method fails in 'parallel_pool_futures' #918

Closed nplee closed 6 days ago

nplee commented 3 weeks ago

This update to the parallel_pool_futures function to fall back on the spawn multiprocessing context method if the default fork method fails to run, which is most likely to happen on MacOS, although fortunately hasn't been seen to happen frequently with CIAO, and this update is in response to the first such instance reported through the helpdesk. While spawn is slower than fork, for the cases where parallel_pool_futures is [or can be] used in the CIAO scripts, the additional overhead to set up the parallelization with spawn will still be quicker than running the tasks serially (e.g. calculating the weighted source and background ARFs and RMFs simultaneously rather one-after-the-other).

This update also linted lingering whitespace in the code.

DougBurke commented 6 days ago

We're going to need some sort of a README entry I think.

DougBurke commented 6 days ago

Also I should talk to you to go through this change as I don't understand it. ..