icecube / flarestack

Unbinned likelihood analysis code for astroparticle physics datasets
https://flarestack.readthedocs.io/en/latest/?badge=latest
MIT License
8 stars 7 forks source link

Forking on macOS #277

Open ekun0608 opened 1 year ago

ekun0608 commented 1 year ago

Running the test demo on macOS 10.15.7 (in a conda environment with python 3.10.11) results in an error "The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec(). Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY_YOU_MUST_EXEC() to debug.”

To reproduce it run python -m unittest discover tests/

Apparently fork() is unsafe on macOS, spawn* functions should be used instead.

I solved the problem of forking by adding the line mp.set_start_method('forkserver') below the if __name__ == "__main_": line of multiprocess_wrapper.py.

mlincett commented 1 year ago

Thanks @ekun0608 for reporting. Would you mind providing a semi-complete log including the lines before the error appears?

Maybe @JannisNe has a Mac and can confirm this issue?

ekun0608 commented 1 year ago
.INFO:root:Testing 'fixed_weight' MinimisationHandler class
...
.INFO:root:Testing MinimisationHandler analysis chain
WARNING:flarestack.cluster.submitter:No submitter implemented for host server None! Using LocalSubmitter but you wont't be able to use cluster operations!
INFO:flarestack.core.minimisation:Using 'standard' LLH class
INFO:flarestack.core.injector:Initialising Injector for IC86-2011
INFO:flarestack.core.multiprocess_wrapper:Added 140 trials to queue. Now processing.
INFO:flarestack.core.multiprocess_wrapper:140 tasks remaining.
140
INFO:flarestack.core.minimisation:Using 'standard' LLH class
INFO:flarestack.core.minimisation:Using 'standard' LLH class
INFO:flarestack.core.minimisation:Using 'standard' LLH class
INFO:flarestack.utils.create_acceptance_functions:Saving IC86-2011 acceptance values to: /Users/kunemma/flarestack/flarestack/analyses/stacking/scratch/flarestack__data/input/acceptance_functions/all_sky_3_year/IC86-2011.pkl
The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.
The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.
The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().

and then the same lines from "Break on" to "You MUST exec()." many many times. Modules before that work well (Testing get_diffuse_flux util functions. Testing get_rates util functions.)