search

fritzsedlazeck / Sniffles

Structural variation caller using third generation sequencing
Other
556 stars 92 forks source link

TypeError: cannot pickle '_thread.RLock' object on macOS #512

Open madagiurgiu25 opened 10 hours ago

madagiurgiu25 commented 10 hours ago

Hi,

I have encountered the following error when running sniffles 2.4 (python 3.10) on a macos.

Error:

sniffles --threads 1  --input ../mapping/ngmlr.bam --minsupport 4 --vcf sniffles2.vcf  --allow-overwrite
2024-10-21 15:40:28,332 INFO sniffles.main (64570): Running Sniffles2, build 2.4
2024-10-21 15:40:28,332 INFO sniffles.main (64570):   Run Mode: call_sample
2024-10-21 15:40:28,332 INFO sniffles.main (64570):   Start on: 2024/10/21 15:40:28
2024-10-21 15:40:28,332 INFO sniffles.main (64570):   Working dir: /Users/madag/Projects/PhD/github/test/decoil-pre/tests/examples/sim_1502/sv
2024-10-21 15:40:28,332 INFO sniffles.main (64570):   Used command: /Users/madag/miniforge3/envs/envdecoil_18102024_6/bin/sniffles --threads 1 --input ../mapping/ngmlr.bam --minsupport 4 --vcf sniffles2.vcf --allow-overwrite
2024-10-21 15:40:28,332 INFO sniffles.main (64570): ==============================
2024-10-21 15:40:28,348 INFO sniffles.main (64570): Opening for reading: ../mapping/ngmlr.bam
2024-10-21 15:40:28,349 INFO sniffles.main (64570): Opening for writing: sniffles2.vcf (single-sample, sorted)
2024-10-21 15:40:28,353 INFO sniffles.main (64570): 
2024-10-21 15:40:28,353 INFO sniffles.main (64570): Analyzing 99548 alignments total...
2024-10-21 15:40:28,353 INFO sniffles.main (64570): 
2024-10-21 15:40:28,353 INFO sniffles.worker (64570): Starting worker 0
2024-10-21 15:40:28,363 ERROR sniffles.main (64570): Unhandled error while running sniffles.
Traceback (most recent call last):
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/bin/sniffles", line 524, in <module>
    Sniffles2_Main(processes)
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/bin/sniffles", line 443, in Sniffles2_Main
    p.start()
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/lib/python3.10/site-packages/sniffles/parallel.py", line 496, in start
    self.process.start()
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/lib/python3.10/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/lib/python3.10/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/lib/python3.10/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/lib/python3.10/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Users/madag/miniforge3/envs/envdecoil_18102024_6/lib/python3.10/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.RLock' object

It seems that the issue is the start method for the multiprocessing module in Python, which is set by default on spawn in my case (macOS Ventura, platform.system()=='Darwin'). Changing this to fork fixes the error.

I got fixed by including in site-packages/sniffles/parallel.py:

from multiprocessing import set_start_method
set_start_method("fork")
hermannromanek commented 9 hours ago

Hi @madagiurgiu25

Thank you! We're aware of this (see https://github.com/fritzsedlazeck/Sniffles/commit/a4af9926a4ec8278d28ea6d9382b15908ed51488), and it will be fixed in the next release which we hope to get out really soon!

Can you confirm this version - i.e. with condition on "darwin" - also works for you? We dont have any Macs to test Sniffles on.

Thanks, Hermann

madagiurgiu25 commented 9 hours ago

Hi @hermannromanek,

Yes, it also works like this:

import platform
if platform.system() == 'Darwin':
    from multiprocessing import set_start_method
    set_start_method("fork")

Best, Madalina