macOS py3.8 and higher parallel issues

openkim / kliff

KIM-based Learning-Integrated Fitting Framework for interatomic potentials.

https://kliff.readthedocs.io

GNU Lesser General Public License v2.1

34 stars 20 forks source link

macOS py3.8 and higher parallel issues #9

Closed mjwen closed 3 years ago

mjwen commented 3 years ago

Internally, multiprocessing are used in a few places to run some parts of the code in parallel. Due to the changes introduced to the multiprocessing module for macOS in py3.8, some training script are failing since kimpy objects are not pickable (see https://github.com/openkim/kimpy/issues/7). This will be resolved once we make objects in kimpy pickable.

For now, everything should work fine with py3.7 on macOS.

Linux unaffected.

mjwen commented 3 years ago

To make multiprocessing for kliff work on macOS, one can use the fork start method by adding to the training script:

import multiprocessing as mp
mp.set_start_method("fork")