Ctrl-C doesn't end the program

[GoogleCodeExporter]

I'm using Python 2.7, and my program ignores SIGTERM signals when using 
ThreadPoolExecutor.

After some debugging I found that my main thread is blocked on Thread.join(), 
and it holds the interruption until the call returns.

My solution is to use join(timeout), as it will respect the interruptions and 
allow the program to exit properly.

The next issue I found was that the atexit hook will try to wait for pending 
threads anyway.
If the program ended without waiting on these, I suppose skipping this step is 
probably not too bad either.

Not a great solution, but works for me :)

Original issue reported on code.google.com by pa...@inutilfutil.com on 1 Nov 2013 at 11:57

Attachments:

• futures.patch

[kislyuk]

Any chance of applying this patch? As it stands I have to be very careful to use timeouts and insert try..except KeyboardInterrupt: os._exit() statements around my futures code.

[agronholm]

Is this a 2.x specific problem or does it occur on 3.3 and above too? Unless this is a 2.x issue I don't want to apply patches that are too adventurous and this one certainly is. Otherwise you'll have to convince upstream developers first.

[kislyuk]

Yes, this issue is 2.x specific, due to a change in the implementation of Thread.join(). In 3.x, join() can be interrupted and then killed with a double Ctrl-C. Here is an example:

import time
from concurrent.futures import ThreadPoolExecutor

def sleep(i):
    print("Sleeping for %d" % i)
    time.sleep(i)

with ThreadPoolExecutor(max_workers=2) as executor:
    executor.map(sleep, range(8))

$ python example.py
Sleeping for 0
Sleeping for 1
Sleeping for 2
^C^CSleeping for 3
Sleeping for 4
Sleeping for 5
Sleeping for 6
Sleeping for 7
Traceback (most recent call last):
  File "foo.py", line 9, in <module>
    executor.map(sleep, range(8))
  File "/usr/lib/python2.7/site-packages/concurrent/futures/_base.py", line 604, in __exit__
    self.shutdown(wait=True)
  File "/usr/lib/python2.7/site-packages/concurrent/futures/thread.py", line 133, in shutdown
    t.join()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 949, in join
    self.__block.wait()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 340, in wait
    waiter.acquire()
KeyboardInterrupt

Serenity:~ kislyuk$ python3 foo.py
Sleeping for 0
Sleeping for 1
Sleeping for 2
^CTraceback (most recent call last):
  File "foo.py", line 9, in <module>
    executor.map(sleep, range(8))
  File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/concurrent/futures/_base.py", line 581, in __exit__
    self.shutdown(wait=True)
  File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/concurrent/futures/thread.py", line 139, in shutdown
    t.join()
  File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/threading.py", line 1063, in join
    self._wait_for_tstate_lock()
  File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/threading.py", line 1079, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt
Sleeping for 3
^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/concurrent/futures/thread.py", line 39, in _python_exit
    t.join()
  File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/threading.py", line 1063, in join
    self._wait_for_tstate_lock()
  File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/threading.py", line 1079, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt

You can see that by pressing Ctrl-C twice, I was able to abort the script immediately when run by Python 3, but not 2.

[kislyuk]

Thank you!