detect function: pickling issue

[williams-jacquelinec]

I have been working on the empirical bayes footprint detection example from the Read the Docs tutorial (https://footprint-tools.readthedocs.io/en/latest/tutorials/posterior.html) but ran into an error when trying to run the detect tool command. The error states that there is an issue with pickling in the reduction.py file. From looking online, I've found that pickling lambda functions is not supported by Python 3+. Is there a solution to this issue? Or maybe an updated reduction.py file available?

I have also pasted the error message below:

Traceback (most recent call last):
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/bin/ftd", line 8, in <module>
    sys.exit(main())
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/lib/python3.9/site-packages/footprint_tools/cli/detect.py", line 281, in run
    dl_iter = dl.batch_iter(batch_size=batch_size, num_workers=n_threads)
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/lib/python3.9/site-packages/genome_tools/data/dataset.py", line 44, in batch_iter
    return iter(dp)
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/lib/python3.9/site-packages/genome_tools/data/loaders.py", line 160, in __iter__
    return data_loader_iter(self)
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/lib/python3.9/site-packages/genome_tools/data/loaders.py", line 73, in __init__
    w.start()
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/lib/python3.9/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/lib/python3.9/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/lib/python3.9/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Users/jacquelinewilliams/opt/anaconda3/envs/footprint-tools/lib/python3.9/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'deviation_stats.__init__.<locals>.<lambda>'

[ExplorerKut]

https://stackoverflow.com/questions/25348532/can-python-pickle-lambda-functions Check this post it may help.

[jvierstra]

Have you found a work around? This works for me using miniconda and the included YML enviromental config file...

[shu-z]

I am having the same issue.. I tried both pip install and using conda with the YML file. Do you have any suggestions for workarounds?