Closed pkothiyal closed 4 years ago
Greetings @pkothiyal.
A 100x coverage dataset is going to require much more than 12G of memory per sample. If you don't have a machine with hundreds of gigabytes of RAM, you'll need to do some error correction on your reads, and perhaps even consider downsampling to 40x-50x coverage. The kevlar documentation has additional recommendations.
I hope this helps!
Thanks @standage
I'll try it on a down-sampled trio first.
However, the error message I saw earlier does not mention low memory and lists high FPR as the bottleneck. Can I run the workflow on 100x coverage with 64G per sample and expect to not see the high FPR error or could that be related to something other than allocated memory? Thanks.
However, the error message I saw earlier does not mention low memory and lists high FPR as the bottleneck.
I think you're failing to see the relationship between memory and accuracy (as measured by FPR) described in the documentation I linked in my previous comment. Insufficient memory is the reason the FPR is too high. There are two approaches to fixing this problem: 1) reduce the information content of the reads, or 2) increase memory. Most of the k-mers in a high coverage data set will be the product of a sequencing error. Correcting these errors will drastically reduce the number of k-mers that have to be tracked/queried, and thus the memory required for high accuracy (low FPR). On the other hand, if your machine has lots of RAM you can simply allocate a larger chunk for each sample to increase the k-mer count accuracy (reduce the FPR).
Thank you - that makes sense. I'll run it on our cluster with more memory per sample and will also test it on down-sampled data.
Hello,
We are testing kevlar on a single human trio for de novo mutation calling. We originally had Illumina BAM files which we converted to fastq to run kevlar. This is high coverage data (~100x) and I don't believe any error correction was performed. I ran the test using the example snakemake workflow and kept all parameters the same except assigned 12G memory to each trio member. I used 64 cores for the job.
Looks like the job is failing due to high FPR in the case (~0.95). Before I play around with max_fpr setting for case, I wanted to see if I'm missing something more fundamental here. Looking at the config file, I just noticed that I should also change mean coverage from 30 to 100!
Would appreciate any feedback on how I can complete this test run. Thank you for your help. Prachi
Error log: