Erratic logging behavior with ProcessPool

[visrra]

ProcessPool seems to display outputs from its tasks only the first time it is executed (tested on JupyterLab). It doesn't display outputs after that unless the notebook is refreshed or the number of nodes of ProcessPool is changed.

Sample code:

from pathos.multiprocessing import ProcessPool

def worker1(x):
    print('end')

pool = ProcessPool(nodes=2)

a = pool.map(lambda x: worker1(x), list(range(4)))

I overcame this by using pathos.helpers.mp, but it would be nice to have the issue resolved in ProcessPool.

[mmckerns]

Sorry for the excessively slow reply here, but I'm not seeing this issue. It sounds like, however, if you are only seeing print after the notebook is refreshed or the nodes are changed, then for whatever reason you are seeing the effects of the pool singleton. You can clear the pool singleton by calling the clear method of the pool object. The normal workflow is:

>>> from pathos.pools import ProcessPool
>>> pool = ProcessPool(2)
>>> pool.map(lambda x:x*2, range(2,4))
[4, 6]
>>> pool.close()
>>> pool.join()
>>> pool.clear() # delete the pool singleton

Please reopen if there's more to add to the issue.