search

lindermanlab / ssm

Bayesian learning and inference for state space models
MIT License
569 stars 198 forks source link

Numerical stability? #52

Closed ShawnGeller closed 5 years ago

ShawnGeller commented 5 years ago

Thank you for building this great package.

I have more of a question rather than an issue: It's stated in Rabiner's review on hidden markov models that the naive forward-backward message passing algorithms are numerically unstable, are you able to avoid this problem just by taking logs first?

slinderman commented 5 years ago

That's right, the naive message passing algorithms compute in probability space, and products of probabilities can go to zero very quickly. We're working in log probability space, which incurs a bit of extra cost but gives numerical stability. I should also note that there are algorithms that work in probability space but renormalize the messages at each step. These would probably also suffice, and might be a tad bit faster, but I haven't implemented them.

ShawnGeller commented 5 years ago

Thank you for the clarification!