Continuous input distributions

[trappmartin]

Hi, @ams939 and I just realized that there are no continuous input nodes. We would add codes for Gaussian distributions. Are there any things we should keep in mind when doing so?

Thanks, Martin

[trappmartin]

@khosravipasha Do you have any suggestions on how to best code up the GPU related bits?

[khosravipasha]

Hi, Adding Guassian should be relatively straightforward. Mainly need to keep track of sufficient statisitics needed during training. We all the flow function to accumulate the sufficent statistics needed, also need to multiply them by the flow value which is basically probability of getting to that input node given input x. The formula would be similar to equition 7 from einsum network paper (difference is if you have missing values during training we have a special case for that, need to also track "missing flow").

I guess for guassian with fixed sigma, only need to track the sum node_flow and sum node_flow * value (and if might have missing data during training also want to add up flow for missing values, see example for binomial here, should be very similar for Gaussian

https://github.com/Juice-jl/ProbabilisticCircuits.jl/blob/27cb093439c8db5b6e59f75567800ff92d4fffa6/src/nodes/binomial_dist.jl#L70-L79

For GPU learning we need the following functions implemented:

• For each input type we have two Strcuts, one is for cpu version and one is for gpu (eg. Binomial vs BitsBinomial)
• bits: converts the input node into bits (i.e. continigous arrays) for paramters and temporary heap memory for keeping track of required suffiecient statistics for that input node.
• unbits: does the opposite and reads from the heap and moves it to cpu into Julia structs.
• flow: for the bits version of input nodes
• update_params: this does the EM update (either mini batch for full batch) with some intertia.

let me know if you run into any other issues, will be happy to help.