marwan116
commented
4 years ago I thought a workaround to the problem, if it is too hard to fix this, would be to disable the decorator and the following workaround works if the value passed toenabled is not a callable
@wrapt.decorator(enabled=False)
def dummy_decorator(wrapped, instance, args, kwargs):
print("before wrapped call")
res = wrapped(*args, **kwargs)
print("after wrapped call")
return resthen the below won't code throw any error
with Parallel(n_jobs=2, prefer="processes") as parallel:
fs = parallel(delayed(add)(x=x, y=y) for x, y in zip([1, 2], [3, 4]))
for f in fs:
print(f)however - to dynamically set enabled, I thought using a callable _enabled would be the way to go and this sadly doesn't work:
def _enabled():
return False
@wrapt.decorator(enabled=_enabled)
def dummy_decorator(wrapped, instance, args, kwargs):
print("before wrapped call")
res = wrapped(*args, **kwargs)
print("after wrapped call")
return resI see on other issue threads (mainly https://github.com/GrahamDumpleton/wrapt/issues/102) that perhaps an ObjectProxy is picklable/serializable if we explicitly define the __reduce__ and __reduce_ex__ methods - would this be the workaround needed here - i.e. to implement the decorator as a wrapper instead?
GrahamDumpleton
ludaavics
twiddli
I am posting this to share the issue I face when trying to use multiple processes with a function wrapped with a
wrapt.decorator.I am using the following libraries:
wrapt==1.12.1joblib=0.14.1python=3.7.6Please see the dummy example below to reproduce the error
So first I try the code using multithreading and it works fine
I get the following output:
Then when I try to use processes instead of threads:
I get an error mainly:
NotImplementedError: object proxy must define __reduce_ex__()below is the full traceback: