search

lava-nc / lava

A Software Framework for Neuromorphic Computing
https://lava-nc.org
Other
547 stars 140 forks source link

Jupyter tutorials on Windows #95

Open PhilippPlank opened 2 years ago

PhilippPlank commented 2 years ago

The usage of multiprocessing on Windows in combination with jupyter notebooks does not work very well (see links below). Some of our tutorials define the ProcessModels within the jupyter notebook and during run, multiprocessing will try to spawn these models. This will most likely fail with the error AttributeError: Can't get attribute '<proc_model_name>' on <module '__main__' (built-in)> exitcode = _main(fd, parent_sentinel)

The error will also not show in the notebook, but it will just stall.

The issue is quite known and a combination of problems of Windows and IPython, so this will not be solvable. The recommended solution is to put the code which multiprocessing will use into separate modules and import it in the notebook.

Should we change the tutorials to import all their code (we can still show it) or have a Windows warning/error for some of them?

https://medium.com/@grvsinghal/speed-up-your-python-code-using-multiprocessing-on-windows-and-jupyter-or-ipython-2714b49d6fac https://github.com/microsoft/vscode-jupyter/issues/941

awintel commented 2 years ago

Interestingly, this seems to work for some people... What is the difference that makes it work for some? It would be very instructive to keep the PyProcModels in the tutorials.

It does not work for me, but it must have worked for those that created some of the current executable tutorials. Were these never tested on Windows?

PhilippPlank commented 2 years ago

This issue is still present in the 0.4.0 release. We are aware of it and testing different fixes, e.g. PR #259. Such workarounds do solve the issue, but introduce additional code in the juypter tutorial itself.

If anyone has a better solution or idea, feel free to contribute or discuss!

PS: In the longer term perspective we probably move away from the multiprocessing library, which should also solve this issue.

tim-shea commented 7 months ago

@PhilippPlank What is the status of this issue?