search

esa / pygmo2

A Python platform to perform parallel computations of optimisation tasks (global and local) via the asynchronous generalized island model.
https://esa.github.io/pygmo2/
Mozilla Public License 2.0
434 stars 57 forks source link

pickle.loads fails to find module for user-defined problem for ipyparallel_island #22

Closed skyshuford closed 4 years ago

skyshuford commented 4 years ago

Is there a way to use ipyparallel_island archipelago with a user-defined problem I created using modules loaded from my working directory?

archi.wait_check() yields:

RuntimeError: The asynchronous evolution of a pythonic island of type 'Ipyparallel island' raised an error: Traceback (most recent call last): File "C:\Users\Skyler\Anaconda3\lib\site-packages\pygmo\_py_islands.py", line 593, in run_evolve return pickle.loads(ret.get()) File "C:\Users\Skyler\Anaconda3\lib\site-packages\ipyparallel\client\asyncresult.py", line 169, in get raise self.exception() File "C:\Users\Skyler\Anaconda3\lib\site-packages\ipyparallel\client\asyncresult.py", line 226, in _resolve_result raise r ipyparallel.error.RemoteError: ModuleNotFoundError(No module named 'module_containing_udp')

It seems like a pickling issue; is there a work-around?

Problem solves when not using archipelago, and pygmo.rosenbrock solves when I use ipyparallel_island.

bluescarni commented 4 years ago

The problem seems to be that the ipyparallel processes are unable to find your module module_containing_udp. You need to make sure that the ipyparallel cluster is able to locate your module. This can be done by either manipulating the PYTHONPATH variable or simply starting the ipyparallel cluster from a working directory from which your custom module can be found.

skyshuford commented 4 years ago

That was it. Appreciate the speedy response.