search

agronholm / pythonfutures

Backport of the concurrent.futures package to Python 2.6 and 2.7
Other
232 stars 51 forks source link

Attempt to handle KeyboardInterrupt exceptions #50

Open jacksontj opened 8 years ago

jacksontj commented 8 years ago

For #25 there are effectively two problems. (1) the worker process doesn't handle the keyboard interrupt and as such the process pool cannot shutdown properly. c2ae8dc fixes this issue, such that the worker processes handle the exception and continue working. (2) There is no way to terminate a process pool. In the event that a keyboard interrupt was received and you do in fact want to exit-- you have no choice but to wait for the tasks to complete (assuming you have the first fix I mentioned). the addition of a terminate() method (f5727bd) allows you to do a more forcible shutdown of the pool-- clearing all unstarted jobs and forcibly killing all inflight processes.

So to recap c2ae8dc makes it so you can handle the keyboard interrupt from the caller and f5727bd gives you a mechanism to stop the pool.

Potential fix for #25

jacksontj commented 8 years ago

After looking some it seems that python3 has the same issues (unhandled keyboardinterrupt and no terminate() method), so if we think this is a valid fix I can start the ball rolling getting this into python3 as well, but from the issue thread this bug is less of an issue in python3.

agronholm commented 8 years ago

As this is a backport, I only accept fixes that are either backport specific or have been applied upstream as well. API changes are a problem in that regard. And not all upstream fixes are even applicable due to some capabilities being unavailable in older Pythons. But if you get your patch accepted upstream, I'm willing to take another look at it and integrate it into the backport if possible.

Would that first commit work on its own? The KeyboardInterrupt catch I mean?

jacksontj commented 8 years ago

Yes it would, but as my comment states its also a bug upstream. Sounds like I should open the same or upstream and come back to this once that is finalized. The issue exists in python3 as well, you can just work around it because the thread join that is deadlocking is interruptible in python3 On Dec 17, 2015 7:42 PM, "Alex Grönholm" notifications@github.com wrote:

As this is a backport, I only accept fixes that are either backport specific or have been applied upstream as well. API changes are a problem in that regard. And not all upstream fixes are even applicable due to some capabilities being unavailable in older Pythons. But if you get your patch accepted upstream, I'm willing to take another look at it and integrate it into the backport if possible.

Would that first commit work on its own? The KeyboardInterrupt catch I mean?

— Reply to this email directly or view it on GitHub https://github.com/agronholm/pythonfutures/pull/50#issuecomment-165661744 .

jacksontj commented 8 years ago

Upstream issue: http://bugs.python.org/issue25908