Fix numerical issues in skew normal fitting

jonassibbesen / rpvg

Method for inferring path posterior probabilities and abundances from pangenome graph read alignments

MIT License

47 stars 6 forks source link

Fix numerical issues in skew normal fitting #24

Closed jeizenga closed 3 years ago

jeizenga commented 3 years ago

My basic implementations of some of the transcendental functions I used in the fitting algorithm could produce inf or nan for certain inputs. This ultimately led to infinite loops in the fitting routine. I've copied over implementations from scipy source code that have better numerical properties.