search

brendanhasz / probflow

A Python package for building Bayesian models with TensorFlow or PyTorch
http://probflow.readthedocs.io
MIT License
171 stars 17 forks source link

Figure out mac/windows multiprocessing issues #32

Open brendanhasz opened 3 years ago

brendanhasz commented 3 years ago

Getting errors w/ the multiprocessing in DataGenerator for windows builds on Python <3.8, and for Mac + Python >= 3.8.

AttributeError: Can't pickle local object 'DataGenerator.__iter__.<locals>.get_data'

Think it might be due to the switch from default process start method of "fork" to "spawn" modes, see here. But that doesn't explain the windows issues...

Example windows test failure

Example mac test failure

Tried doing multiprocessing.set_start_method("fork") for mac + python 3.8, but then just got RuntimeError: context has already been set (example)...

link89 commented 2 months ago

you have to use set_start_method('fork', force=True)