search

PROBIC / mSWEEP

mSWEEP High-resolution sweep metagenomics using fast probabilistic inference
MIT License
13 stars 2 forks source link

Add option to read in a precomputed likelihood matrix #12

Closed tmaklin closed 2 years ago

tmaklin commented 3 years ago

mSWEEP has the capability to write out the internal likelihood matrix as of https://github.com/PROBIC/mSWEEP/pull/11. Since reading the alignments in and filling the likelihood matrix do not parallelize well (the process is bounded by disk access), precomputing the likelihood matrix can save HPC resources in cases where the alignment files are very large. Therefore, an option should be added to read in a likelihood matrix that has been saved with the --write-likelihood toggle.

tmaklin commented 3 years ago

Turns out this is not straightforward to implement because the likelihoods are stored in a very compressed format. It does not make sense to change the format, either, because the current implementation is quite a bit faster to access and also uses a bit less RAM compared to more naive means.

tmaklin commented 2 years ago

This can, and will be, implemented in v1.6.0 with the changes to the model fitting code that allow adding MPI support.