search

BioInf-Wuerzburg / proovread

PacBio hybrid error correction through iterative short read consensus
MIT License
60 stars 20 forks source link

xargs: perl: terminated by signal 9 #139

Closed anu24s closed 5 years ago

anu24s commented 5 years ago

Hi.. am trying to run proovread on my system but it gets terminated giving this error

------------------------------------------------------------------------------

xargs: perl: terminated by signal 9

------------------------------------------------------------------------------

[Mon May 20 11:21:28 2019] bam2cns unexpectedly returned Exited at 'main' '/home/dcb/AJ/Tools/proovread/bin/proovread', line 1636 Last call 'main::correct_sr_mt' '/home/dcb/AJ/Tools/proovread/bin/proovread', line 868

Can anything be done to remove this error and run it successfully?

Here is the complete logfile

dcb@dcb-OptiPlex-7010:/$ proovread -s /media/dcb/SeagateBackupPlusDrive_evolvulus/evolvulus/final/R1_bc_mapped.fastq -s /media/dcb/SeagateBackupPlusDrive_evolvulus/evolvulus/final/R2_bc_mapped.fastq -l /media/dcb/SeagateBackupPlusDrive_evolvulus/evolvulus/final/bc_chloro_blasr_mapped.fasta -p media/dcb/SeagateBackupPlusDrive_evolvulus/evolvulus/final/evolvulus_proovread_out [Mon May 20 10:51:56 2019] Running proovread-2.14.1 under Perl 5.26.2 [Mon May 20 10:51:56 2019] Reading core config [Mon May 20 10:51:56 2019] Reading command line options [Mon May 20 10:51:57 2019] Logging parameter to evolvulus_proovread_out/evolvul us_proovread_out.parameter.log [Mon May 20 10:51:57 2019] Checking short read files

R1_bc_mapped.fastq Detected FASTQ format Estimated short read length: 150 +-0 Etimated short read quality offset: 33 Estimating approximate number of short reads: 18.2M

R2_bc_mapped.fastq Detected FASTQ format Estimated short read length: 150 +-0 Etimated short read quality offset: 33 Estimating approximate number of short reads: 18.2M

[Mon May 20 10:52:49 2019] Checking required binaries

[ok] samtools-1.9 /usr/local/bin [ok] bwa-proovread /home/dcb/AJ/Tools/proovread/bin/../util/bwa [ok] blastn-2.8.0 /usr/local/bin

[Mon May 20 10:52:49 2019] Running mode: sr [Mon May 20 10:52:49 2019] Preparing long reads

------------------------------------------------------------------------------

perl -I/home/dcb/AJ/Tools/proovread/bin/../lib/ /home/dcb/AJ/Tools/proovread/bi\ n/SeqFilter --fasta --in evolvulus_proovread_out/read-long/evolvulusproovread\ out.fq --line-width 80 --lower-case --out evolvulus_proovread_out/read-long/evo\ lvulus_proovread_out.masked.fa --phred-offset 33 [10:52:53] /home/dcb/AJ/Tools/proovread/bin/SeqFilter-1.06 [10:52:53] --in: evolvulus_proovread_out/read-long/evolvulus_proovread_out.fq [10:52:53] Detected FASTQ format, phred-offset 33 153M [=========================================================>] TTS 00:00:01

------------------------------------------------------------------------------

[10:52:54] Input [10:52:54] Sequences 540 # [10:52:54] Total 76323255 bp [10:52:54] Longest 857721 bp [10:52:54] Shortest 12392 bp [10:52:54] N50 206141 bp [10:52:54] N90 68883 bp

------------------------------------------------------------------------------

[10:52:54] Filtered [10:52:54] Sequences 540 # [10:52:54] Total 76323255 bp [10:52:54] Longest 857721 bp [10:52:54] Shortest 12392 bp [10:52:54] N50 206141 bp [10:52:54] N90 68883 bp

------------------------------------------------------------------------------

------------------------------------------------------------------------------

[Mon May 20 10:52:54 2019] Read IDs don't look like proper PacBio subread IDs. Falling back to running mode: sr-noccs

[Mon May 20 10:52:54 2019] Running task bwa-sr-1 [Mon May 20 10:52:54 2019] Indexing Long reads

------------------------------------------------------------------------------

/home/dcb/AJ/Tools/proovread/bin/../util/bwa/bwa-proovread index evolvulus_proo\ vread_out/read-long/evolvulus_proovread_out.masked.fa evolvulus_proovread_out/r\ ead-long/evolvulus_proovread_out.masked.fa 2>evolvulus_proovread_out/bwa-sr-1/e\ volvulus_proovread_out_bwa.log

------------------------------------------------------------------------------

[Mon May 20 10:53:53 2019] Mapping reads /media/dcb/SeagateBackupPlusDrive_evol vulus/evolvulus/final/R1_bc_mapped.fastq /media/dcb/SeagateBackupPlusDrive_evolv ulus/evolvulus/final/R2_bc_mapped.fastq

------------------------------------------------------------------------------

/home/dcb/AJ/Tools/proovread/bin/SeqChunker --chunk-number 1000 --chunk-step 20\ --chunks-per-step 6 --first-chunk 1 /media/dcb/SeagateBackupPlusDrive_evolvulu\ s/evolvulus/final/R1_bc_mapped.fastq /media/dcb/SeagateBackupPlusDrive_evolvulu\ s/evolvulus/final/R2_bc_mapped.fastq | /home/dcb/AJ/Tools/proovread/bin/../util\ /bwa/bwa-proovread mem -b 20 -l 300 -O 2,1 -Y -a -A 5 -B 11 -E 4,3 -T 2.5 -w \ 40 -k 12 -t 4 -W 20 -D 0 -L 30,30 -y 20 -r 1 evolvulus_proovread_out/read-long/\ evolvulus_proovread_out.masked.fa /dev/fd/0 2>evolvulus_proovread_out/bwa-sr-1/\ evolvulus_proovread_out_bwa.log | samtools view -@ 4 -bS /dev/fd/0 >evolvulus_p\ roovread_out/bwa-sr-1/evolvulus_proovread_out_tmp.bam

------------------------------------------------------------------------------

/home/dcb/AJ/Tools/proovread/bin/../util/SeqChunker/bin/SeqChunker FILE should be located at '/home/dcb/AJ/Tools/proovread/bin/../util/SeqChunker/bin/SeqChunker' Reading /media/dcb/SeagateBackupPlusDrive_evolvulus/evolvulus/final/R1_bc_mapped.fastq Reading /media/dcb/SeagateBackupPlusDrive_evolvulus/evolvulus/final/R2_bc_mapped.fastq

[Mon May 20 11:21:05 2019] Sorting BAM evolvulus_proovread_out/bwa-sr-1/evolvul us_proovread_out.bam

------------------------------------------------------------------------------

samtools sort -m 2G -@ 3 -T evolvulus_proovread_out/bwa-sr-1/evolvulus_proovrea\ d_out_tmp -o evolvulus_proovread_out/bwa-sr-1/evolvulus_proovread_out.bam evolv\ ulus_proovread_out/bwa-sr-1/evolvulus_proovread_out_tmp.bam

------------------------------------------------------------------------------

[bam_sort_core] merging from 0 files and 3 in-memory blocks...

[Mon May 20 11:21:07 2019] Indexing BAM evolvulus_proovread_out/bwa-sr-1/evolvu lus_proovread_out

------------------------------------------------------------------------------

samtools index evolvulus_proovread_out/bwa-sr-1/evolvulus_proovread_out.bam

------------------------------------------------------------------------------

[Mon May 20 11:21:07 2019] Correcting Sequences (6 batches)

------------------------------------------------------------------------------

[Mon May 20 11:21:07 2019] Batch 1/6 perl -I/home/dcb/AJ/Tools/proovread/bin/../lib/ /home/dcb/AJ/Tools/proovread/bi\ n/bam2cns --append 1 --bam evolvulus_proovread_out/bwa-sr-1/evolvulus_proovread\ _out.bam --bin-size 20 --coverage 11.25 --ignore-weak-reads 0 --mask-weak-reads\ 0 --max-ins-length 0 --max-ref-seqs 100 --prefix evolvulus_proovread_out/bwa-s\ r-1/evolvulus_proovread_out.0 --qv-offset 33 --ref evolvulus_proovread_out/read\ -long/evolvulus_proovread_out.fq --ref-offset 0

------------------------------------------------------------------------------

[Mon May 20 11:21:07 2019] Batch 2/6 perl -I/home/dcb/AJ/Tools/proovread/bin/../lib/ /home/dcb/AJ/Tools/proovread/bi\ n/bam2cns --append 1 --bam evolvulus_proovread_out/bwa-sr-1/evolvulus_proovread\ _out.bam --bin-size 20 --coverage 11.25 --ignore-weak-reads 0 --mask-weak-reads\ 0 --max-ins-length 0 --max-ref-seqs 100 --prefix evolvulus_proovread_out/bwa-s\ r-1/evolvulus_proovread_out.1 --qv-offset 33 --ref evolvulus_proovread_out/read\ -long/evolvulus_proovread_out.fq --ref-offset 58328886

------------------------------------------------------------------------------

[Mon May 20 11:21:07 2019] Batch 3/6 perl -I/home/dcb/AJ/Tools/proovread/bin/../lib/ /home/dcb/AJ/Tools/proovread/bi\ n/bam2cns --append 1 --bam evolvulus_proovread_out/bwa-sr-1/evolvulus_proovread\ _out.bam --bin-size 20 --coverage 11.25 --ignore-weak-reads 0 --mask-weak-reads\ 0 --max-ins-length 0 --max-ref-seqs 100 --prefix evolvulus_proovread_out/bwa-s\ r-1/evolvulus_proovread_out.2 --qv-offset 33 --ref evolvulus_proovread_out/read\ -long/evolvulus_proovread_out.fq --ref-offset 88971398

------------------------------------------------------------------------------

[Mon May 20 11:21:07 2019] Batch 4/6 perl -I/home/dcb/AJ/Tools/proovread/bin/../lib/ /home/dcb/AJ/Tools/proovread/bi\ n/bam2cns --append 1 --bam evolvulus_proovread_out/bwa-sr-1/evolvulus_proovread\ _out.bam --bin-size 20 --coverage 11.25 --ignore-weak-reads 0 --mask-weak-reads\ 0 --max-ins-length 0 --max-ref-seqs 100 --prefix evolvulus_proovread_out/bwa-s\ r-1/evolvulus_proovread_out.3 --qv-offset 33 --ref evolvulus_proovread_out/read\ -long/evolvulus_proovread_out.fq --ref-offset 107859064

------------------------------------------------------------------------------

xargs: perl: terminated by signal 9

------------------------------------------------------------------------------

[Mon May 20 11:21:28 2019] bam2cns unexpectedly returned Exited at 'main' '/home/dcb/AJ/Tools/proovread/bin/proovread', line 1636 Last call 'main::correct_sr_mt' '/home/dcb/AJ/Tools/proovread/bin/proovread', line 868

thackl commented 5 years ago

Hi,

thanks for getting in touch. I think the problem is the length of your longest sequences that you are trying to correct (>800kb). I'm guessing it's not pacbio data - nanopore or contigs maybe? Anyhow, my guess is that proovread runs out of memory during the consensus step. When I designed it, I only worked with reads of up to a few 10kb. Never optimized it for larger stuff. And, unfortunately, I don't see an easy fix for that right now.

If you are determined to use proovread, you could chop your longest sequences in <100kb fragments, correct them, and afterward just concatenate them together again. Otherwise, you might want to look into https://github.com/isovic/racon for the job.