PROBIC / mSWEEP

mSWEEP High-resolution sweep metagenomics using fast probabilistic inference
MIT License
13 stars 2 forks source link

Automatically determine `--tol` to avoid numerical stability derived convergence issues on large input #25

Closed tmaklin closed 4 months ago

tmaklin commented 10 months ago

The estimation algorithm from rcgpar will sometimes get stuck oscillating around the solution on very large input because of numerical stability issues in checking the convergence of the gradient descent.

While this can be fixed by the --tol and the parameter by increasing its value, it would be better to adapt the parameter value based on the size of the input, which determines the number of floating point operations that are performed.