Closed mxndrwgrdnr closed 7 years ago
It is supported... however, it's just not yet supported in the "unified" interface.
>>> import pathos
>>> p = pathos.helpers.mp.Pool()
>>> res = p.imap(lambda x:x**2, range(4))
>>> res.next(timeout=1)
0
>>> res.next(timeout=1)
1
>>> res.next(timeout=1)
4
>>> res.next(timeout=1)
9
>>> p.close()
>>> p.join()
The above library (pathos.helpers.mp
), is a direct handle to the multiprocess
library, which was split off from pathos
into it's own package.
If this isn't sufficient of an answer, please reopen the ticket.
The built-in multiprocessing library provides a
timeout
argument fornext()
method of the iterator object returned byPool.imap()
. This is really handy for messy subprocesses. For example, I am usingimap
to spawn Chrome instances using Selenium, which when working with a large number of iterations are prone to freezing up. I'm usingPathos
b/c my parallelized function is a class method, which the built-in library cannot handle. But unlessPathos
supports a similar timeout function, I'll probably have to refactor my code to use the built-in library. Any ideas?