I also observed the issue described by timlai4 (program hanging at pid, sts = os.waitpid(self.pid, flag), but it was inconsistent. It seemed like some processes were not joined at the end and left hanging, sporadic deadlocks. Are you on a Unix-based system timlai4? I have not observed any issue with h5py though.
The Multiprocessing doc highlights that the default process start methods are different across the OS. It is specified that the fork start method, which is the Unix-default start method, is unsafe for multithreaded process.
I explicitly specified the spawn start method at the different Pool instantiations. It is difficult to say if it is a fix, since the problem was sporadic. I have tested it multiple times on a Unix system and have not had any issue. I have not tested this fix on other OS, but according to the documentation, spawn was already the method used for Windows and MacOS.
EDIT : Replaces #12, using a specific branch to PR instead of the master branch.
Fixes #9
I also observed the issue described by timlai4 (program hanging at
pid, sts = os.waitpid(self.pid, flag), but it was inconsistent. It seemed like some processes were not joined at the end and left hanging, sporadic deadlocks. Are you on a Unix-based system timlai4? I have not observed any issue with h5py though.The Multiprocessing doc highlights that the default process start methods are different across the OS. It is specified that the
forkstart method, which is the Unix-default start method, is unsafe for multithreaded process.I explicitly specified the
spawnstart method at the different Pool instantiations. It is difficult to say if it is a fix, since the problem was sporadic. I have tested it multiple times on a Unix system and have not had any issue. I have not tested this fix on other OS, but according to the documentation,spawnwas already the method used for Windows and MacOS.EDIT : Replaces #12, using a specific branch to PR instead of the master branch.