search

ruanjue / wtdbg2

Redbean: A fuzzy Bruijn graph approach to long noisy reads assembly
GNU General Public License v3.0
513 stars 94 forks source link

Assembling short read #82

Closed Ivanbh214 closed 5 years ago

Ivanbh214 commented 5 years ago

Hello,

I'm trying to assembling several read from Nanopore tecnology from diferenttens especies of insect for polish after the consensus sequence with poxomis and medaka,I've obtained a consensus sequence file empty. I don't know if meaby with the amplicon size from my samples (140 bp to 600 bp) it's no possible to do this or I've wrong with the parameters. The resutl: ./wtdbg2 -e -x ont -S 1 -g 400 -K 142 -i /home/ivan/wtdbg2/BC25.fastq -t8 -fo wtdbg2 -o /home/ivan/Escritorio/wtdbg -l 142 -m 142

WARNING: unused command-line arguments. For multiple input files, please apply multiple -i. WARNING: try to recognize and add to input files list

total memory 8050832.0 kB available 5660340.0 kB 8 cores Starting program: ./wtdbg2 -e -x -S 1 -g 400 -K 142 -i /home/ivan/wtdbg2/BC25.fastq -t8 -fo wtdbg2 -o /home/ivan/Escritorio/wtdbg -l 142 -m 142 ont pid 4833 date Sat Mar 16 09:15:22 2019

[Sat Mar 16 09:15:22 2019] loading reads 1352 reads [Sat Mar 16 09:15:22 2019] filtering from 1352 reads (>=0 bp), 483640 bp. Try selecting 20000 bp [Sat Mar 16 09:15:22 2019] Done, 78 reads (>=0 bp), 20224 bp, 78 bins PROC_STAT(0) : real 0.020 sec, user 0.000 sec, sys 0.000 sec, maxrss 988.0 kB, maxvsize 84944.0 kB [Sat Mar 16 09:15:22 2019] Set --edge-cov to 3 KEY PARAMETERS: -k 0 -p 21 -K 142.000000 -S 1.000000 -s 0.050000 -g 400 -X 50.000000 -e 3 -L 0 [Sat Mar 16 09:15:22 2019] generating nodes, 8 threads [Sat Mar 16 09:15:22 2019] indexing bins[0,78] (19968 bp), 8 threads [Sat Mar 16 09:15:22 2019] - scanning kmers (K0P21S1.00) from 78 bins 78 bins **** Kmer Frequency **

** 1 - 201 ** Quatiles: 10% 20% 30% 40% 50% 60% 70% 80% 90% 95% 1 1 3 13 26 39 46 48 52 54 If the kmer distribution is not good, please kill me and adjust -k, -p, and -K Cannot get a good distribution anyway, should adjust -S -s, also -A -e in assembly PROC_STAT(0) : real 0.120 sec, user 0.050 sec, sys 0.120 sec, maxrss 92720.0 kB, maxvsize 1404792.0 kB [Sat Mar 16 09:15:22 2019] - Total kmers = 4017 [Sat Mar 16 09:15:22 2019] - average kmer depth = 10 [Sat Mar 16 09:15:22 2019] - 3083 low frequency kmers (<2) [Sat Mar 16 09:15:22 2019] - 0 high frequency kmers (>142) [Sat Mar 16 09:15:22 2019] - indexing 934 kmers, 10131 instances (at most) 78 bins [Sat Mar 16 09:15:22 2019] - indexed 934 kmers, 10131 instances [Sat Mar 16 09:15:22 2019] - masked 0 bins as closed [Sat Mar 16 09:15:22 2019] - sorting PROC_STAT(0) : real 0.120 sec, user 0.050 sec, sys 0.120 sec, maxrss 92720.0 kB, maxvsize 1404792.0 kB [Sat Mar 16 09:15:22 2019] Done 77 reads|total hits 1298 PROC_STAT(0) : real 0.120 sec, user 0.050 sec, sys 0.120 sec, maxrss 92720.0 kB, maxvsize 1404792.0 kB [Sat Mar 16 09:15:22 2019] sorting rdhits ... Done [Sat Mar 16 09:15:22 2019] clipping ... 2.56% bases [Sat Mar 16 09:15:22 2019] generating regs ... 0 [Sat Mar 16 09:15:22 2019] sorting regs ... Done [Sat Mar 16 09:15:22 2019] generating intervals ... 0 intervals [Sat Mar 16 09:15:22 2019] selecting important intervals from 0 intervals [Sat Mar 16 09:15:22 2019] Intervals: kept 0, discarded 0 PROC_STAT(0) : real 0.120 sec, user 0.050 sec, sys 0.120 sec, maxrss 92720.0 kB, maxvsize 1404792.0 kB [Sat Mar 16 09:15:22 2019] Done, 0 nodes [Sat Mar 16 09:15:22 2019] output "/home/ivan/Escritorio/wtdbg.1.nodes". Done. [Sat Mar 16 09:15:22 2019] median node depth = 0 [Sat Mar 16 09:15:22 2019] masked 0 high coverage nodes (>200 or <3) [Sat Mar 16 09:15:22 2019] masked 0 repeat-like nodes by local subgraph analysis [Sat Mar 16 09:15:22 2019] generating edges [Sat Mar 16 09:15:22 2019] Done, 1 edges [Sat Mar 16 09:15:22 2019] output "/home/ivan/Escritorio/wtdbg.1.reads". Done. [Sat Mar 16 09:15:22 2019] output "/home/ivan/Escritorio/wtdbg.1.dot.gz". Done. [Sat Mar 16 09:15:22 2019] graph clean [Sat Mar 16 09:15:22 2019] rescued 0 low cov edges [Sat Mar 16 09:15:22 2019] deleted 0 binary edges [Sat Mar 16 09:15:22 2019] deleted 0 isolated nodes [Sat Mar 16 09:15:22 2019] cut 0 transitive edges [Sat Mar 16 09:15:22 2019] output "/home/ivan/Escritorio/wtdbg.2.dot.gz". Done. [Sat Mar 16 09:15:22 2019] deleted 0 isolated nodes [Sat Mar 16 09:15:22 2019] output "/home/ivan/Escritorio/wtdbg.3.dot.gz". Done. [Sat Mar 16 09:15:22 2019] cut 0 branching nodes [Sat Mar 16 09:15:22 2019] deleted 0 isolated nodes [Sat Mar 16 09:15:22 2019] building unitigs [Sat Mar 16 09:15:22 2019] [Sat Mar 16 09:15:22 2019] output "/home/ivan/Escritorio/wtdbg.frg.nodes". Done. [Sat Mar 16 09:15:22 2019] generating links [Sat Mar 16 09:15:22 2019] generated 1 links [Sat Mar 16 09:15:22 2019] output "/home/ivan/Escritorio/wtdbg.frg.dot.gz". Done. [Sat Mar 16 09:15:22 2019] rescue 0 weak links [Sat Mar 16 09:15:22 2019] deleted 0 binary links [Sat Mar 16 09:15:22 2019] cut 0 transitive links [Sat Mar 16 09:15:22 2019] remove 0 boomerangs [Sat Mar 16 09:15:22 2019] detached 0 repeat-associated paths [Sat Mar 16 09:15:22 2019] remove 0 weak branches [Sat Mar 16 09:15:22 2019] cut 0 tips [Sat Mar 16 09:15:22 2019] pop 0 bubbles [Sat Mar 16 09:15:22 2019] cut 0 tips [Sat Mar 16 09:15:22 2019] output "/home/ivan/Escritorio/wtdbg.ctg.dot.gz". Done. [Sat Mar 16 09:15:22 2019] building contigs [Sat Mar 16 09:15:22 2019] searched 0 contigs [Sat Mar 16 09:15:22 2019] Estimated: [Sat Mar 16 09:15:22 2019] output 0 contigs [Sat Mar 16 09:15:22 2019] Program Done PROC_STAT(TOTAL) **: real 0.220 sec, user 0.170 sec, sys 0.190 sec, maxrss 98556.0 kB, maxvsize 1404792.0 kB

./wtpoa-cns -t8 -i /home/ivan/Escritorio/wtdbg/16-3/wtdbg.ctg.lay.gz -fo wtdbg_consensus.ctg.fa

Thank you very much in advantage

ruanjue commented 5 years ago

Don't filter any reads, try ./wtdbg2 -e -x -S 1 -L 0 -i /home/ivan/wtdbg2/BC25.fastq -t8 -fo wtdbg2 -o /home/ivan/Escritorio/wtdbg -l 1024 -m 100 ont.

Ivanbh214 commented 5 years ago

Thank you for your answer Now with ./wtdbg2 -e -x -S 1 -L 0 -i /home/ivan/wtdbg2/BC25.fastq -t8 -fo wtdbg2 -o /home/ivan/Escritorio/wtdbg -l 1024 -m 100 ont.

Don't take kmers frecuency and the final fastafile is empty as well.I don't know which is the problem. WARNING:

unused command-line arguments. For multiple input files, please apply multiple -i. WARNING: try to recognize and add to input files list

total memory 8050832.0 kB available 6188080.0 kB 8 cores Starting program: ./wtdbg2 -e -x -S 1 -L 0 -i /home/ivan/wtdbg2/BC25.fastq -t8 -fo wtdbg2 -o /home/ivan/Escritorio/wtdbg -l 1024 -m 100 ont pid 15934 date Sat Mar 16 11:12:11 2019 [Sat Mar 16 11:12:11 2019] loading reads 1352 reads [Sat Mar 16 11:12:11 2019] Done, 1352 reads (>=0 bp), 483640 bp, 1348 bins PROC_STAT(0) : real 0.018 sec, user 0.000 sec, sys 0.000 sec, maxrss 856.0 kB, maxvsize 84940.0 kB [Sat Mar 16 11:12:11 2019] Set --edge-cov to 3 KEY PARAMETERS: -k 0 -p 21 -K 1000.049988 -S 1.000000 -s 0.050000 -g 0 -X 50.000000 -e 3 -L 0 [Sat Mar 16 11:12:11 2019] generating nodes, 8 threads [Sat Mar 16 11:12:11 2019] indexing bins[0,1348] (345088 bp), 8 threads [Sat Mar 16 11:12:11 2019] - scanning kmers (K0P21S1.00) from 1348 bins 1348 bins **** Kmer Frequency **

** 1 - 201 ** Quatiles: 10% 20% 30% 40% 50% 60% 70% 80% 90% 95% 2 2 8 22 200 430 600 680 706 740 If the kmer distribution is not good, please kill me and adjust -k, -p, and -K Cannot get a good distribution anyway, should adjust -S -s, also -A -e in assembly PROC_STAT(0) : real 0.118 sec, user 0.020 sec, sys 0.040 sec, maxrss 58808.0 kB, maxvsize 1031000.0 kB [Sat Mar 16 11:12:11 2019] - high frequency kmer depth is set to 1000 [Sat Mar 16 11:12:11 2019] - Total kmers = 31952 [Sat Mar 16 11:12:11 2019] - average kmer depth = 7 [Sat Mar 16 11:12:11 2019] - 0 low frequency kmers (<2) [Sat Mar 16 11:12:11 2019] - 0 high frequency kmers (>1000) [Sat Mar 16 11:12:11 2019] - indexing 31952 kmers, 227848 instances (at most) 1348 bins [Sat Mar 16 11:12:11 2019] - indexed 31952 kmers, 227848 instances [Sat Mar 16 11:12:11 2019] - masked 0 bins as closed [Sat Mar 16 11:12:11 2019] - sorting PROC_STAT(0) : real 0.118 sec, user 0.020 sec, sys 0.040 sec, maxrss 58808.0 kB, maxvsize 1031000.0 kB [Sat Mar 16 11:12:11 2019] Done 1347 reads|total hits 0 PROC_STAT(0) : real 0.819 sec, user 4.760 sec, sys 0.260 sec, maxrss 98000.0 kB, maxvsize 1367520.0 kB [Sat Mar 16 11:12:12 2019] sorting rdhits ... Done [Sat Mar 16 11:12:12 2019] clipping ... 100.00% bases [Sat Mar 16 11:12:12 2019] generating regs ... 0 [Sat Mar 16 11:12:12 2019] sorting regs ... Done [Sat Mar 16 11:12:12 2019] generating intervals ... 0 intervals [Sat Mar 16 11:12:12 2019] selecting important intervals from 0 intervals [Sat Mar 16 11:12:12 2019] Intervals: kept 0, discarded 0 PROC_STAT(0) : real 0.819 sec, user 4.760 sec, sys 0.260 sec, maxrss 98000.0 kB, maxvsize 1367520.0 kB [Sat Mar 16 11:12:12 2019] Done, 0 nodes [Sat Mar 16 11:12:12 2019] output "/home/ivan/Escritorio/wtdbg.1.nodes". Done. [Sat Mar 16 11:12:12 2019] median node depth = 0 [Sat Mar 16 11:12:12 2019] masked 0 high coverage nodes (>200 or <3) [Sat Mar 16 11:12:12 2019] masked 0 repeat-like nodes by local subgraph analysis [Sat Mar 16 11:12:12 2019] generating edges [Sat Mar 16 11:12:12 2019] Done, 1 edges [Sat Mar 16 11:12:12 2019] output "/home/ivan/Escritorio/wtdbg.1.reads". Done. [Sat Mar 16 11:12:12 2019] output "/home/ivan/Escritorio/wtdbg.1.dot.gz". Done. [Sat Mar 16 11:12:12 2019] graph clean [Sat Mar 16 11:12:12 2019] rescued 0 low cov edges [Sat Mar 16 11:12:12 2019] deleted 0 binary edges [Sat Mar 16 11:12:12 2019] deleted 0 isolated nodes [Sat Mar 16 11:12:12 2019] cut 0 transitive edges [Sat Mar 16 11:12:12 2019] output "/home/ivan/Escritorio/wtdbg.2.dot.gz". Done. [Sat Mar 16 11:12:12 2019] deleted 0 isolated nodes [Sat Mar 16 11:12:12 2019] output "/home/ivan/Escritorio/wtdbg.3.dot.gz". Done. [Sat Mar 16 11:12:12 2019] cut 0 branching nodes [Sat Mar 16 11:12:12 2019] deleted 0 isolated nodes [Sat Mar 16 11:12:12 2019] building unitigs [Sat Mar 16 11:12:12 2019] [Sat Mar 16 11:12:12 2019] output "/home/ivan/Escritorio/wtdbg.frg.nodes". Done. [Sat Mar 16 11:12:12 2019] generating links [Sat Mar 16 11:12:12 2019] generated 1 links [Sat Mar 16 11:12:12 2019] output "/home/ivan/Escritorio/wtdbg.frg.dot.gz". Done. [Sat Mar 16 11:12:12 2019] rescue 0 weak links [Sat Mar 16 11:12:12 2019] deleted 0 binary links [Sat Mar 16 11:12:12 2019] cut 0 transitive links [Sat Mar 16 11:12:12 2019] remove 0 boomerangs [Sat Mar 16 11:12:12 2019] detached 0 repeat-associated paths [Sat Mar 16 11:12:12 2019] remove 0 weak branches [Sat Mar 16 11:12:12 2019] cut 0 tips [Sat Mar 16 11:12:12 2019] pop 0 bubbles [Sat Mar 16 11:12:12 2019] cut 0 tips [Sat Mar 16 11:12:12 2019] output "/home/ivan/Escritorio/wtdbg.ctg.dot.gz". Done. [Sat Mar 16 11:12:12 2019] building contigs [Sat Mar 16 11:12:12 2019] searched 0 contigs [Sat Mar 16 11:12:12 2019] Estimated: [Sat Mar 16 11:12:12 2019] output 0 contigs [Sat Mar 16 11:12:12 2019] Program Done PROC_STAT(TOTAL) **: real 0.919 sec, user 5.080 sec, sys 0.290 sec, maxrss 98000.0 kB, maxvsize 1367520.0 kB

Thank you

ruanjue commented 5 years ago

Have a try with '--node-len 512 --node-ovl 0' and also '-l 512 -m 50'.

Ivanbh214 commented 5 years ago

Hello, Thank you for your answer I've tried both to the same time both option and separitily, no resutl consensus sequency empty. I don't understand where could be the problem. I've tried ./wtdbg2 -e -x -S 1 -L 0 -i /home/ivan/wtdbg2/BC25.fastq -t8 -fo wtdbg2 -o /home/ivan/Escritorio/wtdbg_prueba/wtdbg --node-len 512 --node-ovl 0 -l 512 -m 50 ont. and ./wtdbg2 -e -x -S 1 -L 0 -i /home/ivan/wtdbg2/BC25.fastq -t8 -fo wtdbg2 -o /home/ivan/Escritorio/wtdbg_prueba/wtdbg --node-len 512 --node-ovl 0 and ./wtdbg2 -e -x -S 1 -L 0 -i /home/ivan/wtdbg2/BC25.fastq -t8 -fo wtdbg2 -o /home/ivan/Escritorio/wtdbg_prueba/wtdbg -l 512 -m 50 ont. and nothing the consensun.fasta empty

The command line for generate the consensus.fasta ./wtpoa-cns -t 16 -i wtdbg.ctg.lay.gz -fo wtdbg.ctg.fa

I obtein all the file generated with data except the files with extension 1.nodes , .events , frg.nodes and .closed_bins in every test are 0. Could be this a sign for know where is the problem?

Do you have any idea where could be the problem? Any advise ?

In have amplicons from different length 140 bp to 600 bp BC25.txt

Thank you very much

ruanjue commented 5 years ago

Sorry, wtdbg2 cannot assemble those short reads, for it requires read length of at least 4 bins (4*256bp). Please have a try with other assemblers.