It would be very nice to create a simple decorator people could use to submit jobs, as many distributed processing libraries have: People could decorate functions they want to offload to a pipelinin queue for distributed processing. I have something similar in my Liveplots package: https://github.com/fccoelho/liveplots/blob/master/src/liveplots/xmlrpcserver.py
def enqueue(f):
"""Decorator that places the call on a queue"""
def queued(self,*args,**kw):
Q.put((f,(self,)+args))
queued.__doc__ = f.__doc__
queued.__name__ = f.__name__
return queued
where Q is a Queue object.
Also a customized map/reduce (this could be lifted from other python map/reduce libraries) framework would see great adoption.
It would be very nice to create a simple decorator people could use to submit jobs, as many distributed processing libraries have: People could decorate functions they want to offload to a pipelinin queue for distributed processing. I have something similar in my Liveplots package: https://github.com/fccoelho/liveplots/blob/master/src/liveplots/xmlrpcserver.py
where Q is a Queue object.
Also a customized map/reduce (this could be lifted from other python map/reduce libraries) framework would see great adoption.