search

wiremind / remoulade

A fast and reliable distributed task processing library for Python 3.
GNU Lesser General Public License v3.0
39 stars 16 forks source link

Implement a proper timeout per actor #65

Open cp2587 opened 5 years ago

cp2587 commented 5 years ago

The current mechanism to kill an actor is not properly working leading to potential actor running indefinitely. This is very dangerous as potential deadlock are always a possibility and each actor should not have to implement it's own timeout/kill mechanism.
How can we improve the library so it can kill a long-running actor ?

antoinerabany commented 5 years ago

Yes. The issue is that a thread cannot be killed (source). So we either need to use multiprocessing or libraries like greenlet maybe. I need to document more about those and how it is done in other librairies. But I'm alway open if you have any advice.

cp2587 commented 5 years ago

yes, implement a lightwieht coroutine library: it will prove as effective as thread for I/O intensive task. If you use gevent, you need to have compliant code, that's why thread is better in that sense. In python 3, i am not sure gevent is still the canonical way to do it (since standard librairy offer async.io). We should think carefully. @mehdigmira any thought ?