search

superphy / prairiedog

next-gen pangenome graphs for predictive genomics
Other
0 stars 0 forks source link

Have to graph Kmers as they are parsed or we run out of ram #31

Closed kevinkle closed 5 years ago

kevinkle commented 5 years ago 
2019-05-18 15:06:46 prairiedog[22876] DEBUG Counted 2239292 unique Kmers in file Kmers for file samples/SRR2407780.fasta with 44 contigs and K size 11
2019-05-18 15:06:46 prairiedog[22884] DEBUG Counted 2253270 unique Kmers in file Kmers for file samples/SRR3664995.fasta with 71 contigs and K size 11
2019-05-18 15:06:46 prairiedog[22873] DEBUG Counted 2256018 unique Kmers in file Kmers for file samples/SRR2566890.fasta with 52 contigs and K size 11
2019-05-18 15:06:48 prairiedog[22882] DEBUG Counted 2204657 unique Kmers in file Kmers for file samples/SRR3664915.fasta with 60 contigs and K size 11
2019-05-18 15:06:49 prairiedog[22875] DEBUG Counted 2224721 unique Kmers in file Kmers for file samples/SRR3665031.fasta with 46 contigs and K size 11
2019-05-18 15:06:49 prairiedog[22872] DEBUG Counted 2290873 unique Kmers in file Kmers for file samples/SRR2407723.fasta with 56 contigs and K size 11
2019-05-18 15:06:49 prairiedog[22844] DEBUG Calculating n (total number of unique kmers/file)
2019-05-18 15:06:49 prairiedog[22844] DEBUG Calculated n as 5099664087
2019-05-18 15:06:50 prairiedog[22844] DEBUG Initializing NumPy arrays to length 5099664087
2019-05-18 15:07:37 prairiedog[22844] INFO Parsed Kmers for all 2260 files
kevinkle commented 5 years ago

We're pickling them while running with snakemake, so it gets around this issue