Converting BNs to factor graphs or MRFs

henripal commented 7 years ago

Hi all! I've been working with the library for the past couple days and am finding it very useful!

A feature that I would love seeing is the ability to convert easily between Bayesian Network representations to factor graphs or Markov Random Fields. The conversions are not computationally intensive and can sometimes allow to solve certain problems more elegantly (for ex LBP on MRFs only has one type of message rather than Pi and Lambda messages).

Let me know if that is of interest to you and I would happily start working on a PR

mykelk commented 7 years ago

Yes, definitely! We would welcome a PR.

tawheeler commented 7 years ago

We currently do not have an MRF representation. It would be possible to create a MarkovNets.jl package or similar, and it would likely share many things in common with BayesNets.jl. If this is something you are interested in we can discuss.

What did you have in mind?

henripal commented 7 years ago

I was thinking of reproducing what pgmpy does here with both FG and MRF types within BayesNets.jl. It would indeed maybe make more sense to create a new package - and it would indeed share many things with BayesNets.jl. However I'm a little unsure of the best way to proceed since in terms of I-map expressivity FGs > BNs > MRFs ?

mykelk commented 7 years ago

Maybe create FactorGraphs.jl and have BayesNets.jl use it?

henripal commented 7 years ago

How about this - I try to come up with something in my fork of BayesNets.jl and then we can look at it and figure out where to go from there. I'm planning to start with MRFs and essentially copy your design choices (MRF contains a LightGraph.jl undirected graph). Let me know if you want to discuss further or have any advice!

mykelk commented 7 years ago

Sounds good to me.

tawheeler commented 7 years ago

Perfect

henripal commented 7 years ago

Hey, so I created an MRF representation, as well as a couple of helper functions, methods, and the associated Gibbs sampler - all heavily influenced by your structure. It currently only supports discrete distributions. I have an example notebook up with some Ising models and image denoising here and you can take a look at the branch here. Let me know how you think is best to go from here!

tawheeler commented 7 years ago

Thank you. I just wanted to let you know that I will check this on Monday.

hpoit commented 7 years ago

I want to contribute but am just trying to keep afloat at the moment.

tawheeler commented 7 years ago

Great work! The notebook is really, really cool. I posted an issue on TikzGraphs regarding rendering undirected graphs.

Some comments:

having Base.rand(Φ::Factor)::Assignment is great
MRF is nice and short but could make it difficult to beginners to understand. Maybe MarkovNet or 'FactorGraph?
I like that functions are being used for both BNs and MRFs. It may be possible to have overlap between MRFSamplers and BayesNetSamplers, inference methods, etc.

To discuss:

Do we think MRFs and BayesNets should share one supertype? Maybe PGM / ProbabilisticGraphicalModel?
We may want to organize the file structure so that the general PGM interface is defined top-level and then we have separate folders for BNs and MRFs
The package name may need to be changed, or we could create a PGM package that provides the general interface and then have BayesNets.jl and MarkovNets.jl both use it

henripal commented 7 years ago

Thanks! Yes, MarkovNets.jl is better and echoes nicely BayesNets.jl.

I think having a general, abstract GraphicalModels.jl or PGM.jl interface and type and having the dispatches implemented in the MNs.jl and BNs.jl sounds like the way to go - just like what the DifferentialEquations.jl people are doing here or the JuliaML people in their LearnBase.jl?

@hpoit 's Markov Logic Networks could benefit from this framework and the conversions between representations could be implemented at the GraphicalModels.jl level?

hpoit commented 7 years ago

Considering myself a beginner, MRF doesn't bother me at all, specially because that's how Wikipedia first defines it (so I praise the consistency). If I landed on a Julia BN or MRF package today for the very first time, what would really help is the domain-specific language proposed by Dahua Lin, which I think goes along with your discussion point of a top-level defined PGM interface.

henripal commented 7 years ago

@hpoit, yes. See the nice implementation of a similar DSL in Turing.jl

hpoit commented 7 years ago

... or JuMP.jl for the matter of DSL, considered the 'jewel' of Julia.

There is still Deep BN, Deep MRF, and Deep MLN that can be built on top of the work (or platform?) you guys are doing (and I hope we), so it all adds up.

tawheeler commented 7 years ago

I am not sure we need a domain-specific language at this point. I think the interface would be more along the lines of the POMDPs.jl top-level module, which defines functions but does not implement them. If one of you has something in mind for the DSL we could discuss it though.

Hmm. I don't think the top level would have access to either BayesNets.jl or MarkovNets.jl, so defining conversions there might be tricky. That being said, defining conversions within each package would be tricky too. Maybe a PGMUtils package or similar? I am not really happy with that either.

It would like to have a working release of the BayesNets before Spring quarter. Maybe we should decide what can be completed before then and what should be done at a later time?

hpoit commented 7 years ago

Sounds good to me Tim. I am reviewing every line of your notebook and code in general to see how and if I can jump in. So how would the super-type and file structure look like? Why would you need a super-type? Perhaps I could work on the active trail function?

henripal commented 7 years ago

@tawheeler agreed, DSL not on the table at this point. Ok to make a list of most important items to complete before spring Q and start working on it. Should we discuss in this issue or elsewhere?

tawheeler commented 7 years ago

It'd be great to meet in person. I'll email you. @hpoit, I'll organize with you separately

hpoit commented 7 years ago

Henri, I got this from your notebook Tim, we can work on MRF later.

tawheeler commented 7 years ago

I created the slim branch and ProbabilisticGraphicalModels.jl

@hamzaelsaawy We would like to use your Factors.jl as well.

The current plan is:

Factors.jl as a standalone
ProbabilisticGraphicalModels.jl as an interface-defining package (very light)
MarkovNets.jl using Factors & ProbabilisticGraphicalModels
BayesNets.jl using all three (so we can define convert)

henripal commented 7 years ago

created an empty MarkovNets.jl - will start working on filling it out next week!

@hpoit it is normal you can't access MRF unless you checked out the 'graphs' branch from my fork of BayesNets.

hpoit commented 7 years ago

Thanks Henri. How do I Pkg.checkout("https://github.com/henripal/BayesNets.jl/tree/graphs")?

tawheeler commented 7 years ago

Note to posterity: We will put convert in BayesNets, which will use both ProbabilisticGraphicalModels.jl and MarkovNets.jl

hpoit commented 7 years ago

Will convert to MarkovNets.jl pretty much just transform all directed edges into undirected?

henripal commented 7 years ago

@hpoit shoot me an email and I'll walk you through it. first name dot palacci at gmail.

tawheeler commented 7 years ago

I fixed ProbabilsiticGraphicalModels.jl and BayesNets.jl (master branch) so that they are compatible. All tests pass.

juliohm commented 5 years ago

Any update on the PGM package and related implementation packages? It would be nice to have something like pgmpy in Julia :100:

sisl / BayesNets.jl

Converting BNs to factor graphs or MRFs #54