Push notification of new task

[vanatteveldt]

The original design is a pull system, i.e. workers pull the server periodically to ask if there is something to do. If we want, we could add a list of worker pids per task or per server, and use these to wake up the workers on adding a new document to the queue. Below is some test code that shows how this could be done:

import signal
import time
import os

class SleepInterrupted(Exception):
    pass

def handler(signum, frame):
    raise SleepInterrupted()

signal.signal(signal.SIGCONT, handler)

pid = os.getpid()
while True:
    print("Sleeping, feel free to wake me up with 'kill -s SIGCONT {pid}' ..."
          .format(**locals()))
    try:
        time.sleep(10)
    except SleepInterrupted:
        pass
    print("Polling queue")

[vanatteveldt]

I'll just wontfix myself here :)