funkelab / daisy

Block-wise task scheduling for large nD volumes.
MIT License
25 stars 16 forks source link

TypeError: Pickling an AuthenticationString object is disallowed for security reasons #54

Open griffbad opened 6 days ago

griffbad commented 6 days ago

Hey,

I'm trying to update an old script for daisy 0.2.1. to 1.1.1. I (probably naively) attempted to just use my old process function as the new process function in a daisy Task, but the error log gives me the error in the title.

Would love any suggestions, happy to give more as needed, full traceback here:

tornado.application - ERROR - Exception in callback functools.partial(<function wrap.<locals>.null_wrapper at 0x7f9567cc44d0>, <Task finished coro=<TCPStream._send_message() done, defined at /home/griffin/anaconda3/envs/synsev/lib/python3.7/site-packages/daisy/tcp/tcp_stream.py:64> exception=TypeError('Pickling an AuthenticationString object is disallowed for security reasons')>)
Traceback (most recent call last):
  File "/home/griffin/anaconda3/envs/synsev/lib/python3.7/site-packages/tornado/ioloop.py", line 758, in _run_callback
    ret = callback()
  File "/home/griffin/anaconda3/envs/synsev/lib/python3.7/site-packages/tornado/stack_context.py", line 300, in null_wrapper
    return fn(*args, **kwargs)
  File "/home/griffin/anaconda3/envs/synsev/lib/python3.7/site-packages/tornado/ioloop.py", line 779, in _discard_future_result
    future.result()
  File "/home/griffin/anaconda3/envs/synsev/lib/python3.7/site-packages/daisy/tcp/tcp_stream.py", line 69, in _send_message
    pickled_data = pickle.dumps(message)
  File "/home/griffin/anaconda3/envs/synsev/lib/python3.7/multiprocessing/process.py", line 330, in __reduce__
    'Pickling an AuthenticationString object is '
TypeError: Pickling an AuthenticationString object is disallowed for security reasons
cmalinmayor commented 6 days ago

@pattonw Can you advise? Potentially related: The pickling change that you applied also broke/didn't work on Jan's mac yesterday. We might want to rethink/test further.

pattonw commented 5 days ago

Seems like the Pickling an AuthenticationString error comes from when you're trying to pickle a subprocess. Not entirely sure where this error could be coming from since I think all we really need to pickle is the process function.

Could you include the process function you passing into your daisy.Task?

@cmalinmayor interesting. What was your process function and could you post the error message?

griffbad commented 5 days ago

I'm actually trying to rewrite the predict_blockwise script from Synful from the Funke lab. Maybe the problem is that the process calls an external python script inside of being a function defined in the same document? Just an idea I truly have no idea.

Thanks!

pattonw commented 5 days ago

You should be fine to call an external python script. See https://github.com/funkelab/daisy/tree/master/examples/minimal_example

Could you include the process function you are passing into daisy.Task? If you could it would also be ideal if you could either simplify your rewrite or add complexity to the minimal example until you find the minimal change you need to make to a simple script that results in the error you see and then post it here