search

pronobis / libspn

Library for learning and inference with Sum-product Networks
Other
23 stars 3 forks source link

Query about marginal inference #89

Open anicolson opened 4 years ago

anicolson commented 4 years ago

Hi guys, you are doing a great job with this repo.

I have a task that requires some special requirements.

Say an SPN specifies a joint distribution over the following set of variables: x_1, x_2, x_3, x_4, where each is a continuous variable modelled using a univariate Gaussian leaf.

My first case is simple, some of the variables are corrupted during inference (where C indicates corrupted): x_1, x_2, C, C. For this case, the marginal likelihood is used for the leaves of the corrupted variable. The marginal likelihood is computed from -infinity to +infinity, giving a value of 1 for the corrupted leaves.

The second case is similar to the first case, but with added complexity. As for the first case, some of the variables are corrupted. However, for this case, we know that the true value of the variable exists somewhere between the observed value, and -infinity. Therefore, during inference, I need the marginal likelihood to be computed over the bounds -infinity to the obseved value of the variable. If this does not make sense, I have formulas available to make it more clear.

Would this be possible with LibSPN? I have made this work with other SPN libraries out there, but I want to use libSPN due to DPC-SPN and the heavy utilisation of GPU.

Thanks, Aaron.

jostosh commented 4 years ago

Hi Aaron,

I think what you aim to use is a cumulative distribution function, right? That should be quite doable to implement, thought API-wise I'm not sure how to achieve that most elegantly.

Internally, LibSPN uses tensorflow_probability to compute probabilities at the leaves. Most leaf distributions, including the NormalLeaf, could make use of a log_cdf method to do so. Then somehow by indicating corruptness with another tensor, we should be able to get the SPN to compute the thing you want it to.

I will try to work on such a feature later this week.

anicolson commented 4 years ago

That is perfect, thank you very much!

jostosh commented 4 years ago

@anicolson I remember coming across this CDF idea in a recent paper. Could you point me to it? I will need it to get the naming right

anicolson commented 4 years ago

I, unfortunately, don't know the paper you are speaking of.

This is an early (unfinished) version of the work I am trying to accomplish with SPNs: https://arxiv.org/abs/1910.11969

It is based on the work using GMMs by Cooke et al. https://www.ee.columbia.edu/~dpwe/papers/CookeGJV01-mdasr.pdf

The naming convention that they use, as can be seen in subsection 3.1 is "bounded marginalisation", which I have used in my paper so far.

jostosh commented 4 years ago

Ah, bingo! I already read your paper some weeks ago, just didn't make the connection at first :sweat_smile: nice work! I'll go ahead and read the two once more to be sure that I'm doing the right thing