Closed buihoangphuc412 closed 5 years ago
tseemann
commented
5 years ago Seems you got no alignment, and hence no SNPs.
samtools sort: couldn't allocate memory for bam_mem
samtools sort: couldn't allocate memory for bam_memHow much RAM do you have in your computer? I am guessing 8 GB?
I recently fixed a bug in the RAM calculations. Try --ram 3
buihoangphuc412
commented
5 years ago I updated the snippy and fixed it but still have another error. snippy --cpus 4 --ram 3 --ref 26695.gbk --outdir VN0498 --R1 VN0498-s_2_1_sequence.qf.fastq.gz --R2 VN0498-s_2_2_sequence.qf.fastq.gz --force [14:54:18] This is snippy 4.4.0 [14:54:18] Written by Torsten Seemann [14:54:18] Obtained from https://github.com/tseemann/snippy [14:54:18] Detected operating system: linux [14:54:18] Enabling bundled linux tools. [14:54:18] Found bwa - /home/manager/miniconda3/bin/bwa [14:54:18] Found bcftools - /home/manager/miniconda3/bin/bcftools [14:54:18] Found samtools - /home/manager/miniconda3/bin/samtools [14:54:18] Found java - /home/manager/miniconda3/bin/java [14:54:18] Found snpEff - /home/manager/miniconda3/bin/snpEff [14:54:18] Found samclip - /home/manager/miniconda3/bin/samclip [14:54:18] Found seqtk - /home/manager/miniconda3/bin/seqtk [14:54:18] Found parallel - /home/manager/miniconda3/bin/parallel [14:54:18] Found freebayes - /home/manager/miniconda3/bin/freebayes [14:54:18] Found freebayes-parallel - /home/manager/miniconda3/bin/freebayes-parallel [14:54:18] Found fasta_generate_regions.py - /home/manager/miniconda3/bin/fasta_generate_regions.py [14:54:18] Found vcfstreamsort - /home/manager/miniconda3/bin/vcfstreamsort [14:54:18] Found vcfuniq - /home/manager/miniconda3/bin/vcfuniq [14:54:18] Found vcffirstheader - /home/manager/miniconda3/bin/vcffirstheader [14:54:18] Found gzip - /bin/gzip [14:54:19] Found vt - /home/manager/miniconda3/bin/vt [14:54:19] Found snippy-vcf_to_tab - /home/manager/miniconda3/bin/snippy-vcf_to_tab [14:54:19] Found snippy-vcf_report - /home/manager/miniconda3/bin/snippy-vcf_report [14:54:19] Checking version: samtools --version is >= 1.7 - ok, have 1.9 [14:54:19] Checking version: bcftools --version is >= 1.7 - ok, have 1.9 [14:54:19] Checking version: freebayes --version is >= 1.1 - ok, have 1.3 [14:54:19] Checking version: snpEff -version is >= 4.3 - ok, have 4.3 [14:54:19] Using reference: /home/manager/Snippy/26695.gbk [14:54:19] Treating reference as 'genbank' format. [14:54:19] Will use 4 CPU cores. [14:54:19] Using read file: /home/manager/Snippy/VN0498-s_2_1_sequence.qf.fastq.gz [14:54:19] Using read file: /home/manager/Snippy/VN0498-s_2_2_sequence.qf.fastq.gz [14:54:19] Used --force, will overwrite existing VN0498 [14:54:19] Changing working directory: VN0498 [14:54:19] Creating reference folder: reference [14:54:19] Extracting FASTA and GFF from reference. [14:54:20] Wrote 1 sequences to ref.fa [14:54:20] Wrote 1618 features to ref.gff [14:54:20] Creating reference/snpeff.config [14:54:20] Freebayes will process 7 chunks of 242243 bp, 4 chunks at a time. [14:54:20] Using BAM RG (Read Group) ID: VN0498 [14:54:20] Running: samtools faidx reference/ref.fa 2>> snps.log [14:54:20] Running: bwa index reference/ref.fa 2>> snps.log [14:54:21] Running: mkdir -p reference/genomes && cp -f reference/ref.fa reference/genomes/ref.fa 2>> snps.log [14:54:21] Running: ln -sf reference/ref.fa . 2>> snps.log [14:54:21] Running: ln -sf reference/ref.fa.fai . 2>> snps.log [14:54:21] Running: mkdir -p reference/ref && gzip -c reference/ref.gff > reference/ref/genes.gff.gz 2>> snps.log [14:54:21] Running: snpEff build -c reference/snpeff.config -dataDir . -gff3 ref 2>> snps.log [14:54:30] Running: bwa mem -Y -M -R '@RG\tID:VN0498\tSM:VN0498' -t 4 reference/ref.fa /home/manager/Snippy/VN0498-s_2_1_sequence.qf.fastq.gz /home/manager/Snippy/VN0498-s_2_2_sequence.qf.fastq.gz | samclip --max 10 --ref reference/ref.fa.fai | samtools sort -n -l 0 -T /tmp --threads 4 -m 375M | samtools fixmate -m - - | samtools sort -l 0 -T /tmp --threads 4 -m 375M | samtools markdup -T /tmp -r -s - - > snps.bam 2>> snps.log [M::bwa_idx_load_from_disk] read 0 ALT contigs [samclip] samclip 0.2 by Torsten Seemann (@torstenseemann) [samclip] Loading: reference/ref.fa.fai [samclip] Found 1 sequences in reference/ref.fa.fai [M::process] read 266668 sequences (40000200 bp)... [M::process] read 266668 sequences (40000200 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (56, 118425, 18, 44) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (98, 185, 614) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 1646) [M::mem_pestat] mean and std.dev: (208.04, 214.18) [M::mem_pestat] low and high boundaries for proper pairs: (1, 2162) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (219, 310, 450) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 912) [M::mem_pestat] mean and std.dev: (343.57, 173.02) [M::mem_pestat] low and high boundaries for proper pairs: (1, 1143) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (308, 695, 1127) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2765) [M::mem_pestat] mean and std.dev: (692.17, 392.46) [M::mem_pestat] low and high boundaries for proper pairs: (1, 3584) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (90, 174, 1145) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 3255) [M::mem_pestat] mean and std.dev: (486.88, 620.67) [M::mem_pestat] low and high boundaries for proper pairs: (1, 4310) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::mem_process_seqs] Processed 266668 reads in 40.084 CPU sec, 45.677 real sec [samclip] Processed 100000 records... [samclip] Processed 200000 records... [M::process] read 266668 sequences (40000200 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (61, 118502, 25, 46) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (109, 209, 1572) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 4498) [M::mem_pestat] mean and std.dev: (705.05, 798.02) [M::mem_pestat] low and high boundaries for proper pairs: (1, 5961) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (215, 303, 437) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 881) [M::mem_pestat] mean and std.dev: (334.49, 166.07) [M::mem_pestat] low and high boundaries for proper pairs: (1, 1103) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (315, 866, 1173) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2889) [M::mem_pestat] mean and std.dev: (862.56, 697.53) [M::mem_pestat] low and high boundaries for proper pairs: (1, 3747) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (94, 182, 741) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2035) [M::mem_pestat] mean and std.dev: (301.79, 436.83) [M::mem_pestat] low and high boundaries for proper pairs: (1, 2682) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::mem_process_seqs] Processed 266668 reads in 33.092 CPU sec, 39.251 real sec [samclip] Processed 300000 records... [samclip] Processed 400000 records... [samclip] Processed 500000 records... [M::process] read 266668 sequences (40000200 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (55, 118483, 18, 57) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (129, 240, 1043) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2871) [M::mem_pestat] mean and std.dev: (554.61, 676.67) [M::mem_pestat] low and high boundaries for proper pairs: (1, 3785) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (217, 307, 443) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 895) [M::mem_pestat] mean and std.dev: (339.13, 169.53) [M::mem_pestat] low and high boundaries for proper pairs: (1, 1121) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (409, 850, 1137) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2593) [M::mem_pestat] mean and std.dev: (971.83, 702.62) [M::mem_pestat] low and high boundaries for proper pairs: (1, 3782) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (148, 443, 1481) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 4147) [M::mem_pestat] mean and std.dev: (817.75, 817.05) [M::mem_pestat] low and high boundaries for proper pairs: (1, 5480) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::mem_process_seqs] Processed 266668 reads in 32.896 CPU sec, 39.166 real sec [samclip] Processed 600000 records... [samclip] Processed 700000 records... [samclip] Processed 800000 records... [M::process] read 266668 sequences (40000200 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (61, 118381, 17, 60) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (99, 239, 1077) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 3033) [M::mem_pestat] mean and std.dev: (532.22, 612.21) [M::mem_pestat] low and high boundaries for proper pairs: (1, 4011) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (215, 303, 437) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 881) [M::mem_pestat] mean and std.dev: (334.64, 166.24) [M::mem_pestat] low and high boundaries for proper pairs: (1, 1103) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (282, 614, 971) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2349) [M::mem_pestat] mean and std.dev: (559.06, 338.20) [M::mem_pestat] low and high boundaries for proper pairs: (1, 3038) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (109, 204, 816) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2230) [M::mem_pestat] mean and std.dev: (493.95, 583.76) [M::mem_pestat] low and high boundaries for proper pairs: (1, 2937) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::mem_process_seqs] Processed 266668 reads in 33.148 CPU sec, 39.882 real sec [samclip] Processed 900000 records... [samclip] Processed 1000000 records... [M::process] read 266668 sequences (40000200 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (56, 118263, 18, 59) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (142, 284, 986) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2674) [M::mem_pestat] mean and std.dev: (581.40, 694.35) [M::mem_pestat] low and high boundaries for proper pairs: (1, 3518) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (218, 309, 446) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 902) [M::mem_pestat] mean and std.dev: (340.75, 170.60) [M::mem_pestat] low and high boundaries for proper pairs: (1, 1130) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (549, 1146, 2408) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 6126) [M::mem_pestat] mean and std.dev: (1313.67, 876.94) [M::mem_pestat] low and high boundaries for proper pairs: (1, 7985) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (110, 311, 1141) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 3203) [M::mem_pestat] mean and std.dev: (699.95, 805.12) [M::mem_pestat] low and high boundaries for proper pairs: (1, 4234) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::mem_process_seqs] Processed 266668 reads in 33.560 CPU sec, 40.347 real sec [samclip] Processed 1100000 records... [samclip] Processed 1200000 records... [samclip] Processed 1300000 records... [M::process] read 266668 sequences (40000200 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (58, 118281, 18, 53) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (92, 217, 1060) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2996) [M::mem_pestat] mean and std.dev: (555.12, 681.98) [M::mem_pestat] low and high boundaries for proper pairs: (1, 3964) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (216, 307, 442) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 894) [M::mem_pestat] mean and std.dev: (338.08, 168.30) [M::mem_pestat] low and high boundaries for proper pairs: (1, 1120) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (382, 725, 984) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2188) [M::mem_pestat] mean and std.dev: (592.27, 284.49) [M::mem_pestat] low and high boundaries for proper pairs: (1, 2790) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (82, 156, 844) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2368) [M::mem_pestat] mean and std.dev: (438.27, 619.58) [M::mem_pestat] low and high boundaries for proper pairs: (1, 3130) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::mem_process_seqs] Processed 266668 reads in 33.952 CPU sec, 40.455 real sec [samclip] Processed 1400000 records... [samclip] Processed 1500000 records... [samclip] Processed 1600000 records... [M::process] read 95478 sequences (14321700 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (45, 118433, 19, 52) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (102, 161, 960) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2676) [M::mem_pestat] mean and std.dev: (573.11, 800.86) [M::mem_pestat] low and high boundaries for proper pairs: (1, 3777) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (217, 308, 443) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 895) [M::mem_pestat] mean and std.dev: (338.85, 169.40) [M::mem_pestat] low and high boundaries for proper pairs: (1, 1121) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (514, 670, 1092) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2248) [M::mem_pestat] mean and std.dev: (694.82, 396.18) [M::mem_pestat] low and high boundaries for proper pairs: (1, 2826) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (114, 264, 1372) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 3888) [M::mem_pestat] mean and std.dev: (671.39, 788.52) [M::mem_pestat] low and high boundaries for proper pairs: (1, 5146) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::mem_process_seqs] Processed 266668 reads in 35.640 CPU sec, 43.537 real sec [samclip] Processed 1700000 records... [samclip] Processed 1800000 records... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (25, 42222, 9, 16) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (96, 266, 1281) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 3651) [M::mem_pestat] mean and std.dev: (653.44, 791.53) [M::mem_pestat] low and high boundaries for proper pairs: (1, 4836) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (213, 299, 427) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 855) [M::mem_pestat] mean and std.dev: (328.13, 160.60) [M::mem_pestat] low and high boundaries for proper pairs: (1, 1069) [M::mem_pestat] skip orientation RF as there are not enough pairs [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (188, 273, 1049) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2771) [M::mem_pestat] mean and std.dev: (463.94, 430.48) [M::mem_pestat] low and high boundaries for proper pairs: (1, 3632) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RR [M::mem_process_seqs] Processed 95478 reads in 13.288 CPU sec, 17.681 real sec [samclip] Processed 1900000 records... [main] Version: 0.7.17-r1188 [main] CMD: bwa mem -Y -M -R @RG\tID:VN0498\tSM:VN0498 -t 4 reference/ref.fa /home/manager/Snippy/VN0498-s_2_1_sequence.qf.fastq.gz /home/manager/Snippy/VN0498-s_2_2_sequence.qf.fastq.gz [main] Real time: 308.345 sec; CPU: 256.528 sec [samclip] Total SAM records 1970273, removed 328379, allowed 133968, passed 1641894 [samclip] Header contained 3 lines [samclip] Done. [bam_sort_core] merging from 0 files and 4 in-memory blocks... [bam_sort_core] merging from 0 files and 4 in-memory blocks... [15:00:22] Running: samtools index snps.bam 2>> snps.log [15:00:23] Running: fasta_generate_regions.py reference/ref.fa.fai 242243 > reference/ref.txt 2>> snps.log [15:00:24] Running: freebayes-parallel reference/ref.txt 4 -p 2 -P 0 -C 10 --min-repeat-entropy 1.5 --strict-vcf -q 13 -m 60 --min-coverage 10 -F 0.05 -f reference/ref.fa snps.bam > snps.raw.vcf 2>> snps.log [15:09:38] Running: bcftools view --include 'FMT/GT="1/1" && QUAL>=100 && FMT/DP>=10 && (FMT/AO)/(FMT/DP)>=0' snps.raw.vcf | vt normalize -r reference/ref.fa - | bcftools annotate --remove '^INFO/TYPE,^INFO/DP,^INFO/RO,^INFO/AO,^INFO/AB,^FORMAT/GT,^FORMAT/DP,^FORMAT/RO,^FORMAT/AO,^FORMAT/QR,^FORMAT/QA,^FORMAT/GL' > snps.filt.vcf 2>> snps.log normalize v0.5
options: input VCF file - [o] output VCF file - [w] sorting window size 10000 [n] no fail on reference inconsistency for non SNPs false [q] quiet false [d] debug false [r] reference FASTA file reference/ref.fa
stats: biallelic no. left trimmed : 441 no. right trimmed : 776 no. left and right trimmed : 240 no. right trimmed and left aligned : 0 no. left aligned : 0
total no. biallelic normalized : 1457
multiallelic
no. left trimmed : 0
no. right trimmed : 0
no. left and right trimmed : 0
no. right trimmed and left aligned : 0
no. left aligned : 0
total no. multiallelic normalized : 0
total no. variants normalized : 1457
total no. variants observed : 46853
total no. reference observed : 0Time elapsed: 1.57s
[15:09:43] Running: snpEff ann -noLog -noStats -no-downstream -no-upstream -no-utr -t -c reference/snpeff.config -dataDir . ref snps.filt.vcf > snps.vcf 2>> snps.log [15:09:47] Error running command, check VN0498/snps.log
tseemann
commented
5 years ago So the SNP calling is working, but snpEff is failing.
It is a Java app. It coiuld be RAM problems again.
What does grep 'Xm' $(which snpEff) say?
And java -version?
maingoclanmaingoclan
commented
5 years ago Hello Torsten, I have run snippy and I have quite the same problem. ubuntu@zoonoses-hoant:~/Anh$ snippy --outdir SNP2 --ref H37Rv.fasta --peil WTCHG_689876_70045076_1.fastq.gz --force [10:51:06] This is snippy 3.1 [10:51:06] Written by Torsten Seemann torsten.seemann@gmail.com [10:51:06] Obtained from https://github.com/tseemann/snippy [10:51:06] Detected operating system: linux [10:51:06] Enabling bundled linux tools. [10:51:06] Found bwa - /home/ubuntu/miniconda3/bin/bwa [10:51:06] Found samtools - /home/ubuntu/miniconda3/bin/samtools [10:51:06] Found tabix - /home/ubuntu/miniconda3/bin/tabix [10:51:06] Found bgzip - /home/ubuntu/miniconda3/bin/bgzip [10:51:06] Found snpEff - /home/ubuntu/miniconda3/bin/snpEff [10:51:06] Found java - /home/ubuntu/miniconda3/bin/java [10:51:06] Found gzip - /bin/gzip [10:51:06] Found parallel - /home/ubuntu/miniconda3/bin/parallel [10:51:06] Found freebayes - /home/ubuntu/miniconda3/bin/freebayes [10:51:06] Found freebayes-parallel - /home/ubuntu/miniconda3/bin/freebayes-parallel [10:51:06] Found fasta_generate_regions.py - /home/ubuntu/miniconda3/bin/fasta_generate_regions.py [10:51:06] Found vcfstreamsort - /home/ubuntu/miniconda3/bin/vcfstreamsort [10:51:06] Found vcfuniq - /home/ubuntu/miniconda3/bin/vcfuniq [10:51:06] Found vcffirstheader - /home/ubuntu/miniconda3/bin/vcffirstheader [10:51:06] Found vcf-consensus - /mnt/gvl/apps/linuxbrew/bin/vcf-consensus [10:51:06] Found snippy-vcf_to_tab - /home/ubuntu/miniconda3/bin/snippy-vcf_to_tab [10:51:06] Found snippy-vcf_report - /home/ubuntu/miniconda3/bin/snippy-vcf_report [10:51:06] Found snippy-vcf_filter - /home/ubuntu/miniconda3/bin/snippy-vcf_filter [10:51:06] Using reference: /home/ubuntu/Anh/H37Rv.fasta [10:51:07] Treating reference as 'fasta' format. [10:51:07] Will use 8 CPU cores. [10:51:07] Using read file: /home/ubuntu/Anh/WTCHG_689876_70045076_1.fastq.gz [10:51:07] Deleting all files in existing folder: SNP2 [10:51:07] Changing working directory: SNP2 [10:51:07] Creating reference folder: reference [10:51:07] Extracting FASTA and GFF from reference. [10:51:07] Wrote 1 sequences to ref.fa [10:51:07] Wrote 0 features to ref.gff [10:51:07] Freebayes will process 32 chunks of 140160 bp, 8 chunks at a time. [10:51:07] Using BAM RG (Read Group) ID: snps [10:51:07] Running: samtools faidx reference/ref.fa 2>> snps.log [10:51:08] Running: bwa index reference/ref.fa 2>> snps.log [10:51:12] Running: mkdir reference/genomes && cp -f reference/ref.fa reference/genomes/ref.fa 2>> snps.log [10:51:12] Running: mkdir reference/ref && bgzip -c reference/ref.gff > reference/ref/genes.gff.gz 2>> snps.log [10:51:12] Running: (bwa mem -v 2 -M -R '@RG ID:snps SM:snps' -p -t 8 reference/ref.fa /home/ubuntu/Anh/WTCHG_689876_70045076_1.fastq.gz | samtools view -@ 8 -F 3844 -q 60 -S -b -u -T reference/ref.fa - | samtools sort -O bam -o snps.bam -@ 8 -) 2>> snps.log [10:51:12] Running: samtools index snps.bam 2>> snps.log [10:51:12] Running: samtools depth -q 20 snps.bam | bgzip > snps.depth.gz 2>> snps.log [10:51:12] Running: tabix -s 1 -b 2 -e 2 snps.depth.gz 2>> snps.log [10:51:12] Running: fasta_generate_regions.py reference/ref.fa.fai 140160 > reference/ref.txt 2>> snps.log [10:51:13] Running: freebayes-parallel reference/ref.txt 8 -p 1 -q 20 -m 60 --min-coverage 10 -V -f reference/ref.fa snps.bam > snps.raw.vcf 2>> snps.log [10:51:15] Running: /home/ubuntu/miniconda3/bin/snippy-vcf_filter --minqual 10 --mincov 10 --minfrac 0.9 snps.raw.vcf > snps.filt.vcf 2>> snps.log [10:51:16] Running: cp snps.filt.vcf snps.vcf 2>> snps.log [10:51:16] Running: bgzip -c snps.vcf > snps.vcf.gz 2>> snps.log [10:51:16] Running: tabix -p vcf snps.vcf.gz 2>> snps.log [10:51:16] Running: /home/ubuntu/miniconda3/bin/snippy-vcf_to_tab --gff reference/ref.gff --ref reference/ref.fa --vcf snps.vcf > snps.tab 2>> snps.log [10:51:16] Running: vcf-consensus snps.vcf.gz < reference/ref.fa > snps.consensus.fa 2>> snps.log [10:51:16] Error running command, check SNP2/snps.log
Then I have checked the snps.log file: ubuntu@zoonoses-hoant:~/Anh/SNP2$ cat snps.log
[bwa_index] Pack FASTA... 0.07 sec [bwa_index] Construct BWT for the packed sequence... [bwa_index] 3.15 seconds elapse. [bwa_index] Update BWT... 0.05 sec [bwa_index] Pack forward-only FASTA... 0.03 sec [bwa_index] Construct SA from BWT and Occ... 0.78 sec [main] Version: 0.7.17-r1188 [main] CMD: bwa index reference/ref.fa [main] Real time: 4.232 sec; CPU: 4.082 sec
[E::bwa_set_rg] the read group line contained literal
Parsing: snps.raw.vcf No variants found.
Loading reference: reference/ref.fa Loaded 1 sequences. Loading features: reference/ref.gff Parsing variants: snps.vcf Converted 0 SNPs to TAB format.
Can't locate Vcf.pm in @INC (you may need to install the Vcf module) (@INC contains: /home/ubuntu/minicondaaa3/bin/vcftools/perl /home/ubuntu/miniconda3/lib/site_perl/5.26.2/x86_64-linux-thread-multi /home/ubuntu/miniconda3/lib/site_perl/5.26.2 /home/ubuntu/miniconda3/lib/5.26.2/x86_64-linux-thread-multi /home/ubuntu/miniconda3/lib/5.26.2 .) at /mnt/gvl/apps/linuxbrew/bin/vcf-consensus line 6. BEGIN failed--compilation aborted at /mnt/gvl/apps/linuxbrew/bin/vcf-consensus line 6.
I dont know how to fix this. I tried following this https://vcftools.github.io/examples.html but It still doesnt work. I m totally new to command line. It is the problem. Could you help me pls? What can I do?
buihoangphuc412
commented
5 years ago Hello Torsten, I used the script you gave and this is my result
manager@bl8vbox:~$ grep 'Xm' $(which snpEff)
default_jvm_mem_opts="-Xms512m -Xmx1g"
'-Xm'*)
manager@bl8vbox:~$ java -version
openjdk version "11.0.1" 2018-10-16 LTS
OpenJDK Runtime Environment Zulu11.2+3 (build 11.0.1+13-LTS)
OpenJDK 64-Bit Server VM Zulu11.2+3 (build 11.0.1+13-LTS, mixed mode)
@buihoangphuc412 I am told that snpEff doesn;t work with Java > 1.8 I use this java:
java -version
openjdk version "1.8.0_212-ojdkbuild"
OpenJDK Runtime Environment (build 1.8.0_212-ojdkbuild-04)
OpenJDK 64-Bit Server VM (build 25.212-b04, mixed mode)@maingoclanmaingoclan you are using an old unsupported version of snippy (3.1) Please use 4.x
maingoclanmaingoclan
commented
5 years ago Thank you very much. I have updated and it works. :)
nadiaandreani
commented
5 years ago Hi everybody, I am still struggling too.
Thanks
Nadia
buihoangphuc412
commented
5 years ago Thank you so much Torsten, now I can finish all process without error
tseemann
commented
5 years ago @nadiaandreani what does snippy --check print?
nadiaandreani
commented
5 years ago Hi @tseemann! thanks for helping me with this.
here is the snippy check
(snippy) ubuntu@nadia-server:~$ snippy --check [08:35:17] This is snippy 4.4.0 [08:35:17] Written by Torsten Seemann [08:35:17] Obtained from https://github.com/tseemann/snippy [08:35:17] Detected operating system: linux [08:35:17] Enabling bundled linux tools. [08:35:17] Found bwa - /home/ubuntu/miniconda3/envs/snippy/bin/bwa [08:35:17] Found bcftools - /home/ubuntu/miniconda3/envs/snippy/bin/bcftools [08:35:17] Found samtools - /home/ubuntu/miniconda3/envs/snippy/bin/samtools [08:35:17] Found java - /home/ubuntu/miniconda3/envs/snippy/bin/java [08:35:17] Found snpEff - /home/ubuntu/miniconda3/envs/snippy/bin/snpEff [08:35:17] Found samclip - /home/ubuntu/miniconda3/envs/snippy/bin/samclip [08:35:17] Found seqtk - /home/ubuntu/miniconda3/envs/snippy/bin/seqtk [08:35:17] Found parallel - /home/ubuntu/miniconda3/envs/snippy/bin/parallel [08:35:17] Found freebayes - /home/ubuntu/miniconda3/envs/snippy/bin/freebayes [08:35:17] Found freebayes-parallel - /home/ubuntu/miniconda3/envs/snippy/bin/freebayes-parallel [08:35:17] Found fasta_generate_regions.py - /home/ubuntu/miniconda3/envs/snippy/bin/fasta_generate_regions.py [08:35:17] Found vcfstreamsort - /home/ubuntu/miniconda3/envs/snippy/bin/vcfstreamsort [08:35:17] Found vcfuniq - /home/ubuntu/miniconda3/envs/snippy/bin/vcfuniq [08:35:17] Found vcffirstheader - /home/ubuntu/miniconda3/envs/snippy/bin/vcffirstheader [08:35:17] Found gzip - /bin/gzip [08:35:17] Found vt - /home/ubuntu/miniconda3/envs/snippy/bin/vt [08:35:17] Found snippy-vcf_to_tab - /home/ubuntu/miniconda3/envs/snippy/bin/snippy-vcf_to_tab [08:35:17] Found snippy-vcf_report - /home/ubuntu/miniconda3/envs/snippy/bin/snippy-vcf_report [08:35:17] Checking version: samtools --version is >= 1.7 - ok, have 1.9 [08:35:17] Checking version: bcftools --version is >= 1.7 - ok, have 1.9 [08:35:17] Checking version: freebayes --version is >= 1.1 - ok, have 1.3 [08:35:17] Checking version: snpEff -version is >= 4.3 - ok, have 4.3 [08:35:17] Dependences look good!
i work in environments, and I created this recently with just snippy on it.
N
jdeligt
commented
5 years ago Hi @tseemann I ran into the java problem as well. I'm currently in the process of downgrading my java but I did come across an intersting fix. If you run snpEff without the -t flag in Java 11 it does work. So if you want to support higher java versions that might be something to take a look at in the wrapper (i.e. don't use the flag when snippy detects higher java versions)
tseemann
commented
5 years ago @jdeligt Hmm, that is very odd! But maybe it is a solution. I'll do some testing.
` -t : Use multiple threads (implies '-noStats')
cpplyqz
commented
1 year ago I use snippy 4.6.0 to calling my bacteria snp,but there are no result ,I wonder what to do next ,thanks. snippy --outdir 135080RSH2439 --ref 175001_wt/175001R1qc_shovill.fasta --ctgs 135080RSH2439.fasta [21:19:19] This is snippy 4.6.0 [21:19:19] Written by Torsten Seemann [21:19:19] Obtained from https://github.com/tseemann/snippy [21:19:19] Detected operating system: linux [21:19:19] Enabling bundled linux tools. [21:19:19] Found bwa - /home/lj_cpp/miniconda3/bin/bwa [21:19:19] Found bcftools - /home/lj_cpp/miniconda3/bin/bcftools [21:19:19] Found samtools - /home/lj_cpp/miniconda3/bin/samtools [21:19:19] Found java - /home/lj_cpp/miniconda3/bin/java [21:19:19] Found snpEff - /home/lj_cpp/miniconda3/bin/snpEff [21:19:19] Found samclip - /home/lj_cpp/miniconda3/bin/samclip [21:19:19] Found seqtk - /home/lj_cpp/miniconda3/bin/seqtk [21:19:19] Found parallel - /home/lj_cpp/miniconda3/bin/parallel [21:19:19] Found freebayes - /home/lj_cpp/miniconda3/bin/freebayes [21:19:19] Found freebayes-parallel - /home/lj_cpp/miniconda3/bin/freebayes-parallel [21:19:19] Found fasta_generate_regions.py - /home/lj_cpp/miniconda3/bin/fasta_generate_regions.py [21:19:19] Found vcfstreamsort - /home/lj_cpp/miniconda3/bin/vcfstreamsort [21:19:19] Found vcfuniq - /home/lj_cpp/miniconda3/bin/vcfuniq [21:19:19] Found vcffirstheader - /home/lj_cpp/miniconda3/bin/vcffirstheader [21:19:19] Found gzip - /usr/bin/gzip [21:19:19] Found vt - /home/lj_cpp/miniconda3/bin/vt [21:19:19] Found snippy-vcf_to_tab - /home/lj_cpp/miniconda3/bin/snippy-vcf_to_tab [21:19:19] Found snippy-vcf_report - /home/lj_cpp/miniconda3/bin/snippy-vcf_report [21:19:19] Checking version: samtools --version is >= 1.7 - ok, have 1.17 [21:19:19] Checking version: bcftools --version is >= 1.7 - ok, have 1.17 [21:19:19] Checking version: freebayes --version is >= 1.1 - ok, have 1.3.6 [21:19:20] Checking version: snpEff -version is >= 4.3 - ok, have 5.1 [21:19:20] Checking version: bwa is >= 0.7.12 - ok, have 0.7.17 [21:19:20] Using reference: /home/lj_cpp/data/bacterial/175001_wt/175001R1qc_shovill.fasta [21:19:20] Treating reference as 'fasta' format. [21:19:20] Will use 8 CPU cores. [21:19:20] Using read file: /home/lj_cpp/data/bacterial/snippy_result/135080RSH2439.fasta [21:19:20] Creating folder: 135080RSH2439 [21:19:20] Changing working directory: 135080RSH2439 [21:19:20] Creating reference folder: reference [21:19:20] Extracting FASTA and GFF from reference. [21:19:20] Wrote 1 sequences to ref.fa [21:19:20] Wrote 0 features to ref.gff [21:19:20] Shredding /home/lj_cpp/data/bacterial/snippy_result/135080RSH2439.fasta into pseudo-reads [21:19:26] Wrote 464230 fake 250bp reads (20x, stride 25bp) to fake_reads.fq [21:19:26] Freebayes will process 15 chunks of 10770 bp, 8 chunks at a time. [21:19:26] Using BAM RG (Read Group) ID: 135080RSH2439 [21:19:26] Running: samtools faidx reference/ref.fa 2>> snps.log [21:19:26] Running: bwa index reference/ref.fa 2>> snps.log [21:19:27] Running: mkdir -p reference/genomes && cp -f reference/ref.fa reference/genomes/ref.fa 2>> snps.log [21:19:27] Running: ln -sf reference/ref.fa . 2>> snps.log [21:19:27] Running: ln -sf reference/ref.fa.fai . 2>> snps.log [21:19:27] Running: mkdir -p reference/ref && gzip -c reference/ref.gff > reference/ref/genes.gff.gz 2>> snps.log [21:19:27] Running: bwa mem -Y -M -R '@RG\tID:135080RSH2439\tSM:135080RSH2439' -t 8 reference/ref.fa fake_reads.fq | samclip --max 10 --ref reference/ref.fa.fai | samtools sort -n -l 0 -T /tmp --threads 3 -m 2000M | samtools fixmate -m --threads 3 - - | samtools sort -l 0 -T /tmp --threads 3 -m 2000M | samtools markdup -T /tmp --threads 3 -r -s - - > snps.bam 2>> snps.log samtools: /home/lj_cpp/miniconda3/bin/../lib/libtinfow.so.6: no version information available (required by samtools) samtools: /home/lj_cpp/miniconda3/bin/../lib/libncursesw.so.6: no version information available (required by samtools) samtools: /home/lj_cpp/miniconda3/bin/../lib/libncursesw.so.6: no version information available (required by samtools) [M::bwa_idx_load_from_disk] read 0 ALT contigs samtools: /home/lj_cpp/miniconda3/bin/../lib/libtinfow.so.6: no version information available (required by samtools) samtools: /home/lj_cpp/miniconda3/bin/../lib/libncursesw.so.6: no version information available (required by samtools) samtools: /home/lj_cpp/miniconda3/bin/../lib/libncursesw.so.6: no version information available (required by samtools) samtools: /home/lj_cpp/miniconda3/bin/../lib/libtinfow.so.6: no version information available (required by samtools) samtools: /home/lj_cpp/miniconda3/bin/../lib/libncursesw.so.6: no version information available (required by samtools) samtools: /home/lj_cpp/miniconda3/bin/../lib/libncursesw.so.6: no version information available (required by samtools) [samclip] samclip 0.4.0 by Torsten Seemann (@torstenseemann) [samclip] Loading: reference/ref.fa.fai [samclip] Found 1 sequences in reference/ref.fa.fai [M::process] read 320000 sequences (80000000 bp)... [M::process] read 144230 sequences (36042376 bp)... [M::mem_process_seqs] Processed 320000 reads in 54.641 CPU sec, 10.488 real sec [samclip] Processed 100000 records... [samclip] Processed 200000 records... [M::mem_process_seqs] Processed 144230 reads in 30.053 CPU sec, 4.559 real sec [samclip] Processed 300000 records... [samclip] Processed 400000 records... [main] Version: 0.7.17-r1188 [main] CMD: bwa mem -Y -M -R @RG\tID:135080RSH2439\tSM:135080RSH2439 -t 8 reference/ref.fa fake_reads.fq [main] Real time: 18.439 sec; CPU: 85.693 sec [samclip] Total SAM records 464230, removed 71, allowed 0, passed 464159 [samclip] Header contained 3 lines [samclip] Done. [bam_sort_core] merging from 0 files and 3 in-memory blocks... [bam_sort_core] merging from 0 files and 3 in-memory blocks... [21:19:54] Running: samtools index snps.bam 2>> snps.log [21:19:55] Running: fasta_generate_regions.py reference/ref.fa.fai 10770 > reference/ref.txt 2>> snps.log [21:19:55] Running: freebayes-parallel reference/ref.txt 8 -p 2 -P 0 -C 2 -F 0.05 --min-coverage 10 --min-repeat-entropy 1.0 -q 13 -m 60 --strict-vcf -f reference/ref.fa snps.bam > snps.raw.vcf 2>> snps.log [21:19:55] Running: bcftools view --include 'FMT/GT="1/1" && QUAL>=100 && FMT/DP>=10 && (FMT/AO)/(FMT/DP)>=0' snps.raw.vcf | vt normalize -r reference/ref.fa - | bcftools annotate --remove '^INFO/TYPE,^INFO/DP,^INFO/RO,^INFO/AO,^INFO/AB,^FORMAT/GT,^FORMAT/DP,^FORMAT/RO,^FORMAT/AO,^FORMAT/QR,^FORMAT/QA,^FORMAT/GL' > snps.filt.vcf 2>> snps.log normalize v0.5
options: input VCF file - [o] output VCF file - [w] sorting window size 10000 [n] no fail on reference inconsistency for non SNPs false [q] quiet false [d] debug false [r] reference FASTA file reference/ref.fa
stats: biallelic no. left trimmed : 0 no. right trimmed : 0 no. left and right trimmed : 0 no. right trimmed and left aligned : 0 no. left aligned : 0
total no. biallelic normalized : 0
multiallelic
no. left trimmed : 0
no. right trimmed : 0
no. left and right trimmed : 0
no. right trimmed and left aligned : 0
no. left aligned : 0
total no. multiallelic normalized : 0
total no. variants normalized : 0
total no. variants observed : 0
total no. reference observed : 0Time elapsed: 0.00s
[21:19:55] Running: cp snps.filt.vcf snps.vcf 2>> snps.log [21:19:55] Running: /home/lj_cpp/miniconda3/bin/snippy-vcf_to_tab --gff reference/ref.gff --ref reference/ref.fa --vcf snps.vcf > snps.tab 2>> snps.log [21:19:56] Running: /home/lj_cpp/miniconda3/bin/snippy-vcf_extract_subs snps.filt.vcf > snps.subs.vcf 2>> snps.log [21:19:56] Running: bcftools convert -Oz -o snps.vcf.gz snps.vcf 2>> snps.log [21:19:56] Running: bcftools index -f snps.vcf.gz 2>> snps.log [21:19:56] Running: bcftools consensus --sample 135080RSH2439 -f reference/ref.fa -o snps.consensus.fa snps.vcf.gz 2>> snps.log [21:19:56] Running: bcftools convert -Oz -o snps.subs.vcf.gz snps.subs.vcf 2>> snps.log [21:19:56] Running: bcftools index -f snps.subs.vcf.gz 2>> snps.log [21:19:56] Running: bcftools consensus --sample 135080RSH2439 -f reference/ref.fa -o snps.consensus.subs.fa snps.subs.vcf.gz 2>> snps.log [21:19:56] Running: rm -f snps.subs.vcf.gz snps.subs.vcf.gz.csi snps.subs.vcf.gz.tbi 2>> snps.log [21:19:56] Running: rm -f fake_reads.fq 2>> snps.log [21:19:56] Generating reference aligned/masked FASTA relative to reference: snps.aligned.fa samtools: /home/lj_cpp/miniconda3/bin/../lib/libtinfow.so.6: no version information available (required by samtools) samtools: /home/lj_cpp/miniconda3/bin/../lib/libncursesw.so.6: no version information available (required by samtools) samtools: /home/lj_cpp/miniconda3/bin/../lib/libncursesw.so.6: no version information available (required by samtools) [21:19:57] Marked 0 heterozygous sites with 'n' [21:19:57] Creating extra output files: BED GFF CSV TXT HTML [21:19:57] Identified 0 variants. [21:19:57] Result folder: 135080RSH2439 [21:19:57] Result files: [21:19:57] 135080RSH2439/snps.aligned.fa [21:19:57] 135080RSH2439/snps.bam [21:19:57] 135080RSH2439/snps.bam.bai [21:19:57] 135080RSH2439/snps.bed [21:19:57] 135080RSH2439/snps.consensus.fa [21:19:57] 135080RSH2439/snps.consensus.subs.fa [21:19:57] 135080RSH2439/snps.csv [21:19:57] 135080RSH2439/snps.filt.vcf [21:19:57] 135080RSH2439/snps.gff [21:19:57] 135080RSH2439/snps.html [21:19:57] 135080RSH2439/snps.log [21:19:57] 135080RSH2439/snps.raw.vcf [21:19:57] 135080RSH2439/snps.subs.vcf [21:19:57] 135080RSH2439/snps.tab [21:19:57] 135080RSH2439/snps.txt [21:19:57] 135080RSH2439/snps.vcf [21:19:57] 135080RSH2439/snps.vcf.gz [21:19:57] 135080RSH2439/snps.vcf.gz.csi [21:19:57] Walltime used: 38 seconds [21:19:57] Have a suggestion? Tell me at https://github.com/tseemann/snippy/issues [21:19:57] Done.
I run snippy but have this problem, do you know what is this? snippy --cpus 4 --ram 7 --ref 26695.gbk --outdir mysnps --R1 VN0498-s_2_1_sequence.qf.fastq.gz --R2 VN0498-s_2_2_sequence.qf.fastq.gz [04:32:55] This is snippy 4.4.0 [04:32:55] Written by Torsten Seemann [04:32:55] Obtained from https://github.com/tseemann/snippy [04:32:55] Detected operating system: linux [04:32:55] Enabling bundled linux tools. [04:32:55] Found bwa - /home/manager/miniconda3/bin/bwa [04:32:55] Found bcftools - /home/manager/miniconda3/bin/bcftools [04:32:55] Found samtools - /home/manager/miniconda3/bin/samtools [04:32:55] Found java - /home/manager/miniconda3/bin/java [04:32:55] Found snpEff - /home/manager/miniconda3/bin/snpEff [04:32:55] Found samclip - /home/manager/miniconda3/bin/samclip [04:32:55] Found seqtk - /home/manager/miniconda3/bin/seqtk [04:32:55] Found parallel - /home/manager/miniconda3/bin/parallel [04:32:55] Found freebayes - /home/manager/miniconda3/bin/freebayes [04:32:55] Found freebayes-parallel - /home/manager/miniconda3/bin/freebayes-parallel [04:32:55] Found fasta_generate_regions.py - /home/manager/miniconda3/bin/fasta_generate_regions.py [04:32:55] Found vcfstreamsort - /home/manager/miniconda3/bin/vcfstreamsort [04:32:55] Found vcfuniq - /home/manager/miniconda3/bin/vcfuniq [04:32:55] Found vcffirstheader - /home/manager/miniconda3/bin/vcffirstheader [04:32:55] Found gzip - /bin/gzip [04:32:55] Found vt - /home/manager/miniconda3/bin/vt [04:32:56] Found snippy-vcf_to_tab - /home/manager/miniconda3/bin/snippy-vcf_to_tab [04:32:56] Found snippy-vcf_report - /home/manager/miniconda3/bin/snippy-vcf_report [04:32:56] Checking version: samtools --version is >= 1.7 - ok, have 1.9 [04:32:56] Checking version: bcftools --version is >= 1.7 - ok, have 1.9 [04:32:56] Checking version: freebayes --version is >= 1.1 - ok, have 1.3 [04:32:56] Checking version: snpEff -version is >= 4.3 - ok, have 4.3 [04:32:56] Using reference: /home/manager/Snippy/26695.gbk [04:32:56] Treating reference as 'genbank' format. [04:32:56] Will use 4 CPU cores. [04:32:56] Using read file: /home/manager/Snippy/VN0498-s_2_1_sequence.qf.fastq.gz [04:32:56] Using read file: /home/manager/Snippy/VN0498-s_2_2_sequence.qf.fastq.gz [04:32:56] Creating folder: mysnps [04:32:56] Changing working directory: mysnps [04:32:56] Creating reference folder: reference [04:32:56] Extracting FASTA and GFF from reference. [04:32:58] Wrote 1 sequences to ref.fa [04:32:58] Wrote 1618 features to ref.gff [04:32:58] Creating reference/snpeff.config [04:32:58] Freebayes will process 7 chunks of 242243 bp, 4 chunks at a time. [04:32:58] Using BAM RG (Read Group) ID: mysnps [04:32:58] Running: samtools faidx reference/ref.fa 2>> snps.log [04:32:58] Running: bwa index reference/ref.fa 2>> snps.log [04:32:58] Running: mkdir -p reference/genomes && cp -f reference/ref.fa reference/genomes/ref.fa 2>> snps.log [04:32:58] Running: ln -sf reference/ref.fa . 2>> snps.log [04:32:58] Running: ln -sf reference/ref.fa.fai . 2>> snps.log [04:32:58] Running: mkdir -p reference/ref && gzip -c reference/ref.gff > reference/ref/genes.gff.gz 2>> snps.log [04:32:59] Running: snpEff build -c reference/snpeff.config -dataDir . -gff3 ref 2>> snps.log [04:33:08] Running: bwa mem -Y -M -R '@RG\tID:mysnps\tSM:mysnps' -t 4 reference/ref.fa /home/manager/Snippy/VN0498-s_2_1_sequence.qf.fastq.gz /home/manager/Snippy/VN0498-s_2_2_sequence.qf.fastq.gz | samclip --max 10 --ref reference/ref.fa.fai | samtools sort -n -l 0 -T /tmp --threads 4 -m 875M | samtools fixmate -m - - | samtools sort -l 0 -T /tmp --threads 4 -m 875M | samtools markdup -T /tmp -r -s - - > snps.bam 2>> snps.log [M::bwa_idx_load_from_disk] read 0 ALT contigs [samclip] samclip 0.2 by Torsten Seemann (@torstenseemann) [samclip] Loading: reference/ref.fa.fai [samclip] Found 1 sequences in reference/ref.fa.fai [M::process] read 266668 sequences (40000200 bp)... [M::process] read 266668 sequences (40000200 bp)... [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (56, 118425, 18, 44) [M::mem_pestat] analyzing insert size distribution for orientation FF... [M::mem_pestat] (25, 50, 75) percentile: (98, 185, 614) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 1646) [M::mem_pestat] mean and std.dev: (208.04, 214.18) [M::mem_pestat] low and high boundaries for proper pairs: (1, 2162) [M::mem_pestat] analyzing insert size distribution for orientation FR... [M::mem_pestat] (25, 50, 75) percentile: (219, 310, 450) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 912) [M::mem_pestat] mean and std.dev: (343.57, 173.02) [M::mem_pestat] low and high boundaries for proper pairs: (1, 1143) [M::mem_pestat] analyzing insert size distribution for orientation RF... [M::mem_pestat] (25, 50, 75) percentile: (308, 695, 1127) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2765) [M::mem_pestat] mean and std.dev: (692.17, 392.46) [M::mem_pestat] low and high boundaries for proper pairs: (1, 3584) [M::mem_pestat] analyzing insert size distribution for orientation RR... [M::mem_pestat] (25, 50, 75) percentile: (90, 174, 1145) [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 3255) [M::mem_pestat] mean and std.dev: (486.88, 620.67) [M::mem_pestat] low and high boundaries for proper pairs: (1, 4310) [M::mem_pestat] skip orientation FF [M::mem_pestat] skip orientation RF [M::mem_pestat] skip orientation RR [M::mem_process_seqs] Processed 266668 reads in 32.804 CPU sec, 49.160 real sec samtools sort: couldn't allocate memory for bam_mem samtools sort: couldn't allocate memory for bam_mem [04:33:58] Running: samtools index snps.bam 2>> snps.log [04:33:58] Running: fasta_generate_regions.py reference/ref.fa.fai 242243 > reference/ref.txt 2>> snps.log [04:33:58] Running: freebayes-parallel reference/ref.txt 4 -p 2 -P 0 -C 10 --min-repeat-entropy 1.5 --strict-vcf -q 13 -m 60 --min-coverage 10 -F 0.05 -f reference/ref.fa snps.bam > snps.raw.vcf 2>> snps.log [04:33:59] Running: bcftools view --include 'FMT/GT="1/1" && QUAL>=100 && FMT/DP>=10 && (FMT/AO)/(FMT/DP)>=0' snps.raw.vcf | vt normalize -r reference/ref.fa - | bcftools annotate --remove '^INFO/TYPE,^INFO/DP,^INFO/RO,^INFO/AO,^INFO/AB,^FORMAT/GT,^FORMAT/DP,^FORMAT/RO,^FORMAT/AO,^FORMAT/QR,^FORMAT/QA,^FORMAT/GL' > snps.filt.vcf 2>> snps.log normalize v0.5
options: input VCF file - [o] output VCF file - [w] sorting window size 10000 [n] no fail on reference inconsistency for non SNPs false [q] quiet false [d] debug false [r] reference FASTA file reference/ref.fa
stats: biallelic no. left trimmed : 0 no. right trimmed : 0 no. left and right trimmed : 0 no. right trimmed and left aligned : 0 no. left aligned : 0
Time elapsed: 0.00s
[04:33:59] Running: snpEff ann -noLog -noStats -no-downstream -no-upstream -no-utr -t -c reference/snpeff.config -dataDir . ref snps.filt.vcf > snps.vcf 2>> snps.log [04:34:02] Error running command, check mysnps/snps.log