search

fritzsedlazeck / Sniffles

Structural variation caller using third generation sequencing
Other
559 stars 93 forks source link

Error in Multiple-Samples Calling #312

Open zhqduan opened 2 years ago

zhqduan commented 2 years ago

Hello,

I am using sniffles2 to call SVs from multiple samples with the command: sniffles --input 462samples.snf.list.tsv --vcf 462samples.sniffle2.jointcall.vcf --threads 64 However, it fails with the error message:

....... Calling SVs across 1 samples (1731912517 candidates total)...

86725670/1731912517 candidates processed (5%, 48273/s); 174/194 tasks done; parallel 20/64; 172237 SVs. Sniffles2Main: Shutting down workers Sniffles2 Error: An exception occured in a worker process (see surrounding error messages) (Fatal error, exiting.) Process Process-62: Traceback (most recent call last): File "/storage/yangjianLab/duanzhongqu/miniconda3/envs/sniffles/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/storage/yangjianLab/duanzhongqu/miniconda3/envs/sniffles/lib/python3.7/multiprocessing/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/storage/yangjianLab/duanzhongqu/miniconda3/envs/sniffles/lib/python3.7/site-packages/sniffles/parallel.py", line 196, in Main raise e File "/storage/yangjianLab/duanzhongqu/miniconda3/envs/sniffles/lib/python3.7/site-packages/sniffles/parallel.py", line 193, in Main Main_Internal(proc_id,config,pipe) File "/storage/yangjianLab/duanzhongqu/miniconda3/envs/sniffles/lib/python3.7/site-packages/sniffles/parallel.py", line 331, in Main_Internal pipe.send(["return_combine",result]) File "/storage/yangjianLab/duanzhongqu/miniconda3/envs/sniffles/lib/python3.7/multiprocessing/connection.py", line 206, in send self._send_bytes(_ForkingPickler.dumps(obj)) File "/storage/yangjianLab/duanzhongqu/miniconda3/envs/sniffles/lib/python3.7/multiprocessing/connection.py", line 393, in _send_bytes header = struct.pack("!i", n) struct.error: 'i' format requires -2147483648 <= number <= 2147483647

Do you have any suggestion with this error? Thank you in advance!

Best, Zhongqu

fritzsedlazeck commented 2 years ago

Dear Zhongqu, you need to run sniffles2 per sample. As you can see from the documentation --input expects a bam file. you might want to add --snf to record candidates that will be used for sniffles merge approach.

thx Fritz

wdecoster commented 2 years ago

FWIW you get errors like this one if the data is too large to pickle, which is also what multiprocessing uses when passing data back from threads/processes to the main process.

zhqduan commented 2 years ago

Thanks for your prompt response!

@fritzsedlazeck I'm sorry. I didn't make it clear. I have run sniffles2 and get a .snf file for each sample. The 462samples.snf.list.tsv includes the snf files from 462 samples.

@wdecoster I test the command on ~50 samples and it works well. Do you have any suggestion on the large number of samples? For example, define a higher --minsupport.

Thank you very much!

Zhongqu