noxdafox / pebble

Multi threading and processing eye-candy.
GNU Lesser General Public License v3.0
536 stars 52 forks source link

add running() method to Future #106

Closed dcnieho closed 2 years ago

dcnieho commented 2 years ago

Through Pebble's futures i can distinguish whether a task is done or not, and whether it was cancelled or errored. However, when not done, i cannot find whether it is currently running or only pending. I have worked around this by routing all my tasks through a function that provides the notification that a job starts and keeping track of state using that and the done_callback, but this adds a bunch of complication. Would it be possible to add a running() method to your futures, that for instance sets some running flag in the future right before https://github.com/noxdafox/pebble/blob/706966a1fa96deedd329decbc4a2c2e535dff897/pebble/pool/process.py#L431 gets executed?

noxdafox commented 2 years ago

Hello,

I am not sure what is the issue here. Pebble's Futures inherit from concurrent.future.Future and have a running method.

In [1]: import pebble

In [2]: def function():
   ...:     import time
   ...:     time.sleep(100)
   ...: 

In [3]: pool = pebble.ProcessPool()

In [4]: f = pool.submit(function, None)

In [5]: f.running()
Out[5]: True

The method works as expected.

In [15]: for _ in range(100):
    ...:     f = pool.submit(function, None)
    ...: 

In [16]: f.running()
Out[16]: False
dcnieho commented 2 years ago

@noxdafox thank you very much for having a look. I wasn't aware of the difference between the future returned by processpool's submit and the asyncio support. I see now that the problem is that i use the asyncio support (https://pebble.readthedocs.io/en/latest/#pools-and-asyncio). The asyncio future does not have a running() method. Given how much complexity it'll save if i get a future that does have the running method, it worth it for me to try to redesign my solution to work without the asyncio support.

dcnieho commented 2 years ago

this can be closed i think, as implementing running() for asyncio.Future wouldn't be your responsibility of course. And a simplified design avoiding that did the trick for me nicely.