search

jmschrei / pomegranate

Fast, flexible and easy to use probabilistic modelling in Python.
http://pomegranate.readthedocs.org/en/latest/
MIT License
3.33k stars 588 forks source link

Add support for Dynamic Bayesian Networks #200

Closed fredcallaway closed 6 years ago

fredcallaway commented 7 years ago

DBNs can be seen as a generalization of HMMs that allow multiple states variables, i.e. a factored state. One can transform any DBN into a HMM with linked weights, thus it would be relatively easy to support. If there is already a simple way to do this that I'm missing, please let me know!

jmschrei commented 7 years ago

Can you provide a link to show that any DBN can be turned into a HMM? I know that HMMs are the simplest DBN, but I don't think you can turn an arbitrary DBN into a HMM.

Even though I would like to support them, I don't have plans currently to do so. There is too much other stuff going on for me to think about adding in another big model class which has such a rich feature set.

fredcallaway commented 7 years ago

Can you provide a link to show that any DBN can be turned into a HMM?

This assertion is made in section 15.5 of Artificial Intelligence: A Modern Approach by Norvig and Russell. I can't link to it because it is a copyrighted file but if you email me at fredc@llaway.com, I can suggest how to access the material. Intuitively, the mapping from DBN to HMM is accomplished by making each combination of hidden variable values it's own state. For example, assume we have two hidden states Aand B both with possible values {0 1}, in a DBN. We construct an equivalent HMM where the states are the cartesian product of states in the DBN: A0B0, A0B1, A1B0, A1B1.

The power of a DBN comes from the possibility of constructing a sparse transition matrix. For example, A_t1 may be determined only by A_t0, irrespective of B_t0. Similarly, the emission at state A0B0 may be the product of the independent distributions associated with A0 and B0. I think we can accomplish this in pomegranate with tied state transitions and hierarchical distributions.

This analysis assumes that some variables are always hidden and others are always observed and further that each subset of variables is internally independent. I'm not sure if/how one could generalize to cases in which this assumption does not hold.

Anyway, I won't ask you to implement this. But if I can achieve my goal in pomegranate, I could try to generalize it in a way that would be worthy of being part of the library. If so, I could make a pull request. Would you be amenable to this contribution?

jmschrei commented 7 years ago

That's interesting! I actually have that book so I'll read that section--thanks for the reference. I'd love contributions to the library. If you can get it to work, please make a pull request and I'll gladly review it. I'll let you know if I end up moving in this direction soon.

MaStFAU commented 5 years ago

Are DBNs supported now in pomegranate?

jmschrei commented 5 years ago

No. I closed the issue because, unfortunately, my attention has shifted to the point where I'm not going to be able to add in new models. My primary focus is on fixing bugs, adding in simple features, and documenting those features that currently exist.

MaStFAU commented 5 years ago

Ok, thanks for the quick feedback!