search

gpilab / framework

The GPI framework provides the canvas for graphically assembling algorithms.
Other
20 stars 7 forks source link

"Process" type nodes fail with Python 3.8 #35

Closed borupdaniel closed 4 years ago

borupdaniel commented 4 years ago

Process type nodes (e.g., the "Shepp-Logan" node in gpi_core) are failing with the following error message when running GPI in an env with Python 3.8:

Fri Dec  6 12:06:48 2019 - gpi.node:594 - ERROR - computeRun(): Failed
Fri Dec  6 12:06:48 2019 - gpi.node:595 - ERROR - Traceback (most recent call last):
  File "/Users/m206107/src/framework/lib/gpi/node.py", line 591, in computeRun
    self.nodeCompute_thread.start()
  File "/Users/m206107/src/framework/lib/gpi/functor.py", line 200, in start
    self._proc.start()
  File "/Users/m206107/gpi_stack/envs/gpi_test_py38/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/m206107/gpi_stack/envs/gpi_test_py38/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Users/m206107/gpi_stack/envs/gpi_test_py38/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
    return Popen(process_obj)
  File "/Users/m206107/gpi_stack/envs/gpi_test_py38/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/m206107/gpi_stack/envs/gpi_test_py38/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/m206107/gpi_stack/envs/gpi_test_py38/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Users/m206107/gpi_stack/envs/gpi_test_py38/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle 'PTask' object
borupdaniel commented 4 years ago

This seems to be because Python 3.8 now defaults to spawn instead of fork for new processes, as described here: https://github.com/ansible/ansible/issues/63973. I've got a fix prepared that seems to work in some quick testing.