uqfoundation / pathos

parallel graph management and execution in heterogeneous computing
http://pathos.rtfd.io
Other
1.38k stars 89 forks source link

Can pathos do this? #158

Closed Mradr closed 5 years ago

Mradr commented 5 years ago

Hi, I have the following code:

executor = concurrent.futures.ProcessPoolExecutor(1) future = executor.submit(function, var) future.result()

I like to convert this into using pathos. I don't see any compatibility functions - but I might have miss it. Is this possible to do in pathos? Basically it just runs a function that I pass some information to that I then get the results from.

mmckerns commented 5 years ago

It's planned to a more 1-to-1 match for the concurrent.futures interface. However, until then, the equivalent is something like this:

>>> import pathos.pools as pp
>>> executor = pp.ProcessPool(1)
>>> future = executor.apipe(lambda x:x*x, 5)
>>> future.get()
25
>>> 
mmckerns commented 5 years ago

close if this answers your question

ddelange commented 1 year ago

for future readers: here a solution if you're in an async environment and are looking for a similar implementation:

https://gist.github.com/ddelange/643fbb791b398783c04d1ceb90102163/7c93500752f6876a2f6c5234019eb0f86cb00375#file-executor-py-L57-L63