noxdafox / pebble

Multi threading and processing eye-candy.
GNU Lesser General Public License v3.0
525 stars 49 forks source link

TimeoutError traceback of the concurrent process #91

Closed hayd closed 2 years ago

hayd commented 2 years ago

Firstly, thank you for this library - it's great!

Suppose I have the following code:

import time
import traceback
from concurrent.futures import TimeoutError
from pebble import concurrent

a = c = lambda: None
b = lambda: time.sleep(3)

@concurrent.process(timeout=1)
def abc():
  a()
  b()  # times out
  c()

if __name__ == '__main__':
    try:
        abc().result()
    except TimeoutError as err:
        traceback.print_exception(err, err, err.__traceback__)

# Output:

#  Traceback (most recent call last):
#   File "abc.py", line 18, in <module>
#     abc().result()
#   File "/usr/lib64/python3.8/concurrent/futures/_base.py", line 439, in result
#    return self.__get_result()
#   File "/usr/lib64/python3.8/concurrent/futures/_base.py", line 388, in __get_result
#     raise self._exception
# concurrent.futures._base.TimeoutError: ('Task Timeout', 1)

This traceback doesn't show that the process was sleeping in b() when it timed out.

Is there a way to see that or is this a limitation on concurrent.futures ?

noxdafox commented 2 years ago

Hello,

It's not a limitation of concurrent.futures but rather the fact the parent process cannot infer where the child is stuck. Only thing it can do is realize the results did not come back in time and terminate the process.

If you want to understand where a process is blocked, you can printout/log the execution steps and see which one is the step which does not complete in time.

noxdafox commented 2 years ago

Closing this. Please re-open if you need more info.