navis.mesh() multiprocessing error

[lankiszhang]

Description navis.mesh(nl) throw out a runtime error.

To Reproduce

import os
import navis

path_swc_JRC2018U = "./flywire_swc_JRC2018U"

nl_reg = navis.read_swc(path_swc_JRC2018U)
m_reg = navis.mesh(nl_reg,parallel=True,n_cores=32) # also tried n_cores=4, 2, 1, got the same error.

Example Files: flywire_swc_JRC2018U.zip

Exception has occurred: RuntimeError (note: full exception trace is shown but execution is paused at: )

    An attempt has been made to start a new process before the
    current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

    To fix this issue, refer to the "Safe importing of main module"
    section in https://docs.python.org/3/library/multiprocessing.html

File "E:\Jiajun\macros\navis_mesh_debug.py", line 9, in m_reg = navis.mesh(nl_reg,parallel=True,n_cores=32) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1, in (Current frame) RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

    To fix this issue, refer to the "Safe importing of main module"
    section in https://docs.python.org/3/library/multiprocessing.html

Expected behavior navis.mesh() is expected to convert all neurons in the neuronlist to mesh.

Your system

• NAVis version: 1.6.0
• Python version: Python 3.11.8
• Operating System: Windows

Additional context navis.read_swc(nl) throw the same Runtime error when nl has too many neurons.

[schlegelp]

I don't have a Windows machine but I tried your code with Python 3.11.8 and navis 1.6.0 and it works just fine. I don't think this is an issue with navis but rather a weird interaction between multiprocessing on Windows. See this StackOverflow thread and let me know if that solves your problem.

[schlegelp]

As additional reference: by default navis.read_swc will also use multi-processing if there are many files to load (I think the threshold is 200). That explains why it's crashing too.

[lankiszhang]

@schlegelp Thank you for your help! I did see the thread before, but as a rookie I thought the if __name__ == "__main__" has to be put in the code of navis.mesh() to take effect. Which actually not the case, the following code works:

import navis

path_swc_JRC2018U = "E:/Jiajun/python_project/navis/flywire_swc_JRC2018U"
path_obj_JRC2018U = "E:/Jiajun/python_project/navis/flywire_obj_JRC2018U"

if __name__ == "__main__":
    nl_reg = navis.read_swc(path_swc_JRC2018U)
    m_reg = navis.mesh(nl_reg,parallel=True,n_cores=32)
    navis.write_mesh(m_reg, path_obj_JRC2018U,"obj")