search

HewlettPackard / sandpiper

Implementation of the Loopy Belief Propagation algorithm for Apache Spark
Apache License 2.0
42 stars 17 forks source link

Pairwise factors? #7

Closed analyticalpicasso closed 4 years ago

analyticalpicasso commented 6 years ago

Hi,

This is not exactly an issue. I am using the PairwiseBP algorithm for HTTP log files. But unable to understand the factor values in edges. And in sample data also its not very much clear. Can you please provide a little clarity on how are we getting edge factors?

Thanks

manish-marwah commented 6 years ago

Hi, In most cases, edge factors are values selected by domain experts based on experience, since enough data may not be available to learn the factors. That is, the values in an edge factor are assigned based on correlation between the values of the variables; similarly, a node factor (also called prior) is assigned based on experience.

If there is sufficient data, the factors in an MRF can be learnt from data. However, unfortunately, learning parameters in a MRF is much harder than that in directed models. You can use maximum likelihood estimation but there is no closed form solution. For more details, search for 'parameter learning in Markov random fields'.