pnpnpn / timeout-decorator

Timeout decorator for Python
MIT License
628 stars 94 forks source link

It can't support in windows #45

Open xixiaocheng opened 6 years ago

xixiaocheng commented 6 years ago

Maybe it's the problem that windows didn't support signal well.

But do you have any solution ?

bitranox commented 6 years ago

dear xixiao, timer signals can not be used in Windows at all. You might use this decorator with option use_signals=False, or You might use my fork - there the signals are disabled by default in windows : https://github.com/bitranox/wrapt-timeout-decorator

yours sincerely

Robert

xixiaocheng commented 6 years ago

Thanks! I just use your sample code to execute at pycharm.

Error message: pickle.PicklingError: Can't pickle <function mytest at 0x00000000025E9D68>: it's not the same object as main.mytest

bitranox commented 6 years ago

Dear Xixiao, try not to call if from main - I will change the sample code accordingly. BTW, happy chinese new year

so smt. like :

import time
from wrapt_timeout_decorator import *

@timeout(5)
def mytest(message):
    print(message)
    for i in range(1,10):
        time.sleep(1)
        print('{} seconds have passed'.format(i))

def main():
    mytest('starting')

if __name__ == '__main__':
    main()

yours sincerely

Robert

xixiaocheng commented 6 years ago

I also have a request. add an option message="xxxxxxxxxxxxxxxxxxxx"

If the function timeout. Error message will also show~

bitranox commented 6 years ago

You can already customize the exception message : @timeout(... ,exception_message='something') if You need to do that. But to keep the decorator generic, You should catch the Timeout Exception and there print/log whatever You want.

import time
from wrapt_timeout_decorator import *

@timeout(5)
def mytest(message):
    print(message)
    for i in range(1,10):
        time.sleep(1)
        print('{} seconds have passed'.format(i))

def main():
    try:
        mytest('starting')
    except TimeoutError as exc:
        print('whatever You want')
        # take a closer look using
        # sys.excinfo() or
        # traceback.print_exc()
        # reraise if needed: 
        raise TimeoutError from exc

if __name__ == '__main__':
    main()
xixiaocheng commented 6 years ago

Thank you! Your help is very useful! By the way, what's the difference between the wrapt-timeout-decorator and timeout-decorator

bitranox commented 6 years ago

Dear Xiaxiao, the timeout decorator is using functools for wrapping the function --> see sourcecode:

# pnpnpn timeout decorator:
from functools import wraps

we use wrapt, what is more persistent, especially when You debug in pycharm. Also it provides better support in decorating methods of classes, static methods, etc.:

# wrapt-timeout-decorator
import wrapt

but most important for windows is, that we use "dill + multiprocess" instead of "pickle + multiprocessing" - distinct "multiprocess" != "multiprocessing" !!!

# pnpnpn timeout decorator:
import multiprocessing  # implizitely imports pickle
# wrapt-timeout-decorator
# of course You need to install dill and multiprocess via pip first
# see the links further below
import dill              # the better sibling of pickle
import multiprocess      # the better sibling of multiprocessing, using dill instead pickle

since we dont have signals in Windows, we need to run the decorated function in a seperate process that can be terminated after the timeout occurs (You can not terminate threads). In order to do that, the decorated function have to be "pickled" (serialized).

pickle is somehow limited, dill can serialize more types then pickle. This is important, because if the decorated function or method contains some elements that are not pickable, the decorator will fail with smth, like " ... not pickable " see : https://pypi.python.org/pypi/dill see : https://github.com/uqfoundation/multiprocess see: https://stackoverflow.com/questions/19984152/what-can-multiprocessing-and-dill-do-together

conclusion:

1 You want to use dill instead of pickle, because pickle doesn't support functions with closures, lambdas, or functions in main. (and many more cool things - just read the docs)

2 multiprocessing makes some bad choices about pickling, it can not use dill. Don't get me wrong, it makes some good choices that enable it to pickle certain types so they can be used in a pool's map function. However, since we have dill that can do the pickling, multiprocessing's own pickling becomes a bit limiting. multiprocess also adds some nice features that multiprocessing doesn't have, like multi-args in the map function.

3 as a windows programmer You want to use "dill + multiprocess" instead of "pickle + multiprocessing". I use it also on Linux, since I have not found any disadvantages. (although dill is a little bit slower than pickle - but the difference is negligable unless You pickle tenthousands of objects)

4 You might try :

# try to test Your programs with following imports - should pass without problems
# of course You need to install dill and multiprocess via pip first
import dill as pickle
import multiprocess as multiprocessing

yours sincerely

Robert

UlionTse commented 6 years ago
import time
from timeout_decorator import *

@timeout(5)
def mytest(message):
    print(message)
    for i in range(1,10):
        time.sleep(1)
        print('{} seconds have passed'.format(i))

def main():
    try:
        mytest('starting')
    except TimeoutError:
        print('whatever You want')

if __name__ == '__main__':
    main()

raise error:

  File "C:\Install\Anaconda3\lib\site-packages\timeout_decorator\timeout_decorator.py", line 78, in new_function
    old = signal.signal(signal.SIGALRM, handler)
AttributeError: module 'signal' has no attribute 'SIGALRM'
bitranox commented 6 years ago

You use pnpnpn´s timeout decorator from here - so You need to put use_signals=False on windows, so:

@timeout(5, use_signals=False)
def mytest(message):
    ...

my fork does that automatically, You can look here :

https://github.com/bitranox/wrapt-timeout-decorator

yours sincerely

Robert