How to implement boolean operations/factors for binary random variables, or more generally conditional probability tables?

[DayalStrub]

I'm trying to work through Winn et al.'s MBML book using ForneyLab and am stuck trying to implement the model in Chapter 2. I've been through some of the demos and the documentation, and am sorry if I've missed something obvious.

This seems to boil down to implementing boolean factors and factors representing more general conditional probability tables for binary variables. Is it possible to do something like:

g = FactorGraph()

## prior statistics
p_x = placeholder(:p_x)
p_y = placeholder(:p_y)

## priors
@RV x ~ Bernoulli(p_x)
@RV y ~ Bernoulli(p_y)

## MOCK And factor
@RV z ~ And(x, y) 

placeholder(z)
;

and more generally for factors defined using conditional probability tables?

The model has both an AND factor and an "AddNoise" factor defined as a table, as seen below and in the book.

Many thanks!

[ThijsvdLaar]

@DayalStrub Thanks for your question. ForneyLab currently does not implement updates for boolean logic (we mainly aim at dynamic models), but it should in principle be possible to derive them. Here's my two cents.

A first step would be to translate the probability table to a node function. Let's assume three binary random variables z, y, x \in {0, 1}, which relate through z = x AND y. In general, this node function could be defined as f(z, x, y) = \delta(z - g(x, y)), where \delta encodes a Kronecker delta function, and g a functional representation of the probability table. For example, the AND operator could be encoded as g(x, y) = x*y, and OR as g(x, y) = x + y - x*y.

A second step would be to derive the sum-product messages. For example, \mu_z(z) = \sum_x \sum_y \mu_x(x) \mu_y(y) \delta(z - g(x, y)), where \mu_x(x) = Ber(x | p_x) etc.

With respect to the noise injection, this could be modeled by a Transition node with off-diagonal transition probabilities; e.g. A=[0.9 0.1; 0.1 0.9].

@ismailsenoz What do you think?

[ThijsvdLaar]

In discussion with @ismailsenoz , we though it might be a good idea to write a demo that derives and implements these nodes (AND and OR) as extensions to ForneyLab.

We think functionality related to Boolean logic should not be an integral part of the FL core, but we do want to show how FL can be extended with custom nodes and updates (related to issue #29 ). The AND and OR node might be good candidates for this purpose. The demo should then show how to:

(1) formulate the node functions; (2) derive the sum-product updates; (3) implement the nodes and updates as an addon to FL; (4) use them in an example model.

[DayalStrub]

@ThijsvdLaar thanks for your quick and comprehensive reply!

I must admit that I don't fully understand when you say that the main focus of FL is dynamic models, as I always find it hard to make these distinctions, and imagine one could come up with some convoluted switching state space model that needs Boolean algebra, but I guess the fact that I can't think of an example means you're probably onto something. Plus, I defer to your deeper understanding of the topic.

Some more/node-building examples would be great! I would be happy to contribute, but given I'm new to Julia (and ForneyLab is one of the reasons I'm finally picking it up), I'm afraid I can't be of much help. I look forward to your implementations/examples, and will have a play with FL and try to implement it myself in the meantime, or at least understand the package a bit better.

In point (3), are you suggesting a "ForneyTools" extension, which could even become more of a community maintained package, with lots more nodes, so one doesn't have to re-implement new nodes each time/themselves?

[erlebach]

I have a need for the AND operator in Forneylab. has a demo been written to demonstrate how to extend Forneylab to accomplish this? Thanks.

[ThijsvdLaar]

A "how to build a node" demo is now implemented by #206 in demo/implementing_additional_nodes.ipynb