Can't pickle decorated function

[dbishai]

I am using decorated functions within the DEAP framework to limit the tree size and it seems SCOOP does not support pickling decorated functions.

@apply_decorator
def mutUniform(*args, **kwargs):
    return gp.mutUniform(*args, **kwargs)

@apply_decorator
def mutUniform(*args, **kwargs):
    return gp.mutUniform(*args, **kwargs)

toolbox.register("mutate", mutUniform, expr=toolbox.expr_mut, pset=pset)

output:

[2017-02-21 11:47:39,593] scoopzmq  (b'127.0.0.1:57909') WARNING Pickling Error: Can't pickle <function mutUniform at 0x7f8dc2e7a730>: it's not the same object as __main__.mutUniform
scoop._comm.scoopexceptions.ReferenceBroken: This element could not be pickled: FutureId(worker=b'127.0.0.1:57909', rank=1):partial(0,)=None.
[2017-02-21 11:47:42,399] scoopzmq  (b'127.0.0.1:57909') ERROR   A worker exited unexpectedly. Read the worker logs for more information. SCOOP pool will now shutdown.

My code works fine with the built in Python 3 multiprocessing pool.map function btw.

[joernhees]

hmm, without a mcve there's not really a good way to help you...

• how's apply_decorator, toolbox.expr_mut and pset defined?
• did you intentionally list the first part of the code twice?

that said, from my experience it can help to statically define a composed function instead of loosely coupling it via toolbox... (this means use def myfunc(...) and later toolbox.register('myfunc', myfunc) instead of toolbox.register('myfunc', base_func, some_crazy_param_bindings, making_a_whole_new_function)).