Closed liningbo closed 1 year ago
There is support for the initializer in pathos.multiprocessing._ProcessPool
. Is that what you wanted?
@mmckerns pathos.multiprocessing._ProcessPool is just the alias of multiprocessing.Pool, multiprocessing.Pool do not work well in some multiprocess case, because of serialization problem, but pathos.multiprocessing._ProcessPool do well, so I hope pathos.multiprocessing._ProcessPool can support the initializer parameter. Thank you for reply.
pathos.multiprocessing._ProcessPool
is identical to multiprocess.Pool
, but not multiprocessing.Pool
. The former uses dill
, while the latter uses pickle
. You will note that multiprocess.Pool
(i.e. pathos.multiprocessing._ProcessPool
) supports an initializer and has better serialization.
@mmckerns yes, my last comment make a mistake as you said. I want to say that pathos.multiprocessing.ProcessPool works well , but multiprocessing.Pool do not work well sometime beacase of serialization problem. I hope pathos.multiprocessing.ProcessPool can support an initializer and has better serialization.
To be clear, the initializer argument is supported in pathos
in pathos.multiprocessing._ProcessPool
and in multiprocess
in Pool
. See #138 for a similar issue and response. The interface for pathos.multiprocessing.ProcessPool
may support an initializer in the near future, however.
Sometimes We need to do some Initialization work when start new process, but pathos.multiprocessing.ProcessPool seem not support it. The multiprocessing.Pool has the initializer parameter, so suggest pathos support initializer parameter for pathos.multiprocessing.ProcessPool as multiprocessing.Pool. Thank you.
class Pool(object): ''' Class which supports an async version of applying functions to arguments. ''' _wrap_exception = True