Missing k-mers?

[rwittler]

When building a graph with option -s, i.e., only considering k-mers at least appearing twice, the final number of k-mers considered is different from what I get with other tools. For instance, on the dataset SRR7167957 from the SRA, I get:

Bifrost: 6745326084 ("CompactedDBG::filter(): Found 6745326084 unique k-mers") KMC3: 6752490849 SANS: 6752490849

I used the current version (c156741) and the command that I used is: Bifrost build -v -s input.txt -o reads.bifrost -t 16 --colors

[GuillaumeHolley]

Hi Roland,

Long time no see :)

The log message of CompactesDBG::filter() is only an approximation of the number of unique k-mers in the input dataset and is not the final (exact) number of k-mers in the graph. I agree the message should clearly state it is an estimate and I'll change it in the next release.

Could you confirm the final number of k-mers in the graph with: awk '{if ($1=="S"){SUM+=length($3)-k+1}} END {print SUM}' graph.gfa

Guillaume

[GuillaumeHolley]

Hi @rwittler,

I'll close this for now but don't hesitate to reopen if need be.

Guillaume

[rwittler]

Hi Guillaume,

Thank you very much for your quick answer. Sorry for the late response. My families X-mas break was extended by SARS-Cov.

I tried the awk command that you suggested. It returned 417501438.

I also used the bifrost option to produce a fasta file and counted the 31-mers in there. Both KMC3 and SANS return the same value as above.

But this is quite different from the number of k-mers (appearing at least twice) reported by both KMC3 and SANS on the original input: 6752490849

The values differ by a factor of 16.

Any idea what else I could try?

Thanks, Roland

On 03.01.23 13:30, Guillaume Holley wrote:

Hi @rwittler https://github.com/rwittler,

I'll close this for now but don't hesitate to reopen if need be.

Guillaume

— Reply to this email directly, view it on GitHub https://github.com/pmelsted/bifrost/issues/64#issuecomment-1369714309, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFO7S3GF24YXTRUTY2RQQP3WQQLVDANCNFSM6AAAAAATGZ3IM4. You are receiving this because you were mentioned.Message ID: @.***>

[GuillaumeHolley]

Hi Roland,

No worries :) I'll download the SRR7167957 and let you know what I find.

Guillaume

[GuillaumeHolley]

Hi Roland,

I think you might have misinterpreted the KMC3 output and the correct number of k-mers occurring twice or more in the dataset is indeed 417501438 as reported per Bifrost and KMC3.

First of all, from the SRA website, I could see that the dataset is 8.4 Gbp across 1.8M reads. 8.4 Gbp - 1.8M reads * 30 = 8.346 billion 31-mers in the dataset. Which means that in the absolute best case, there can only be 4.173 billion k-mers occurring twice or more in the dataset which is much less than 6,752,490,849.

I ran Bifrost and from the first lines in the log output:

CompactedDBG::build(): Estimated number of k-mers occurring at least once: 6747567378
CompactedDBG::build(): Estimated number of k-mers occurring twice or more: 407928353

From those lines, I could see that the number of 6,752,490,849 was most likely the number of unique 31-mers in the graph rather than the number of k-mers occurring at least twice. I confirm that I also get 417,501,438 31-mers in my output graph.

Then, I ran KMC3: kmc -v -k31 -ci2 SRR7167957.fastq kmc_SRR7167957 and in the log output was:

...
No. of unique k-mers: 6752490849
No. of unique counted k-mers: 417501438
...

So all is good, KMC3 and Bifrost find the same number of 31-mers :)

Guillaume

[rwittler]

Hi Guillaume,

I am so sorry for bothering you with this. I don't know what has happened exactly, but also SANS reports 407928353 k-mers. So everything is consistent now - probably has been all the time. No problem at all.

Thank you for your patience.

Best, Roland

On 10.01.23 15:38, Guillaume Holley wrote:

Hi Roland,

I think you might have done a little error of interpretation with KMC3 and the correct number of k-mers occurring twice or more in the dataset is indeed 417501438 as reported per Bifrost and KMC3.

First of all, from the SRA website, I could see that the dataset is 8.4 Gbp across 1.8M reads. 8.4 Gbp - 1.8M reads * 30 = 8.346 billion 31-mers in the dataset. Which means that in the absolute best case, there can only be 4.173 billion k-mers occurring twice or more in the dataset which is much less than 6,752,490,849.

I ran Bifrost and from the first lines in the log output:

|CompactedDBG::build(): Estimated number of k-mers occurring at least once: 6747567378 CompactedDBG::build(): Estimated number of k-mers occurring twice or more: 407928353 |

From those lines, I could see that the number of 6,752,490,849 was most likely the number of unique 31-mers in the graph rather than the number of k-mers occurring at least twice. I confirm that I also get 417,501,438 31-mers in my output graph.

Then, I ran KMC3: |kmc -v -k31 -ci2 SRR7167957.fastq kmc_SRR7167957| and in the log output was:

|... No. of unique k-mers: 6752490849 No. of unique counted k-mers: 417501438 ... |

So all is good, KMC3 and Bifrost find the same number of 31-mers :)

Guillaume

— Reply to this email directly, view it on GitHub https://github.com/pmelsted/bifrost/issues/64#issuecomment-1377377702, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFO7S3EW6G63KYTNNWZ62UTWRVX7RANCNFSM6AAAAAATGZ3IM4. You are receiving this because you were mentioned.Message ID: @.***>