apfeffer
commented
8 years ago Hi,
I’m sorry about the slow response. I hope this response helps you.
There’s a history of drawing connections between neural networks and probabilistic models. For example, Boltzmann machines are an example of both simultaneously.
More recently, people look at incorporating NNs in a probabilistic model, but not so much in a generative sense as you describe. One way people use them is to suggest proposals in a MCMC algorithm – the idea is that the NN gives you a pretty good guess of what the right values of some variables are, and that can guide your search, but the overall model is still probabilistic.
If I understand what you’re describing, you want to use PP to learn the weights of a NN. It’s not clear to me what the benefit of this would be. NN weight learning algorithms are highly developed with several excellent and highly scalable implementations. If you just want to learn the weights of an NN, I wouldn’t use PP.
Does this help?
Avi
From: toncho11 [mailto:notifications@github.com] Sent: Saturday, June 25, 2016 3:53 AM To: p2t2/figaro figaro@noreply.github.com Subject: [p2t2/figaro] Modeling a neural network in Figaro (#589)
Bayes nets and neural networks have things in common - for example they are both directed graphical models. I am getting indications that a NN can be modeled using probabilistic programming language.
What we are trying to estimate is the weights for each node that connects to the nodes of the next layer. So Nodes can be represented as random variables whose distribution gives us the weights between the nodes in the ANN. Activation function must be set somewhere. This is how we define a model.
So the idea of training is to set the input and output as observed and then apply reasoning such as EMWithBP.
Once we have these we can apply the model to new inputs (Observed) and infer (in top to bottom direction) to predict the output. The output is a random variable where we say >0.5 is class 1 and <0.5 is class 2.
Is the above correct? Is it similar to the Spam filter example? Here (NN perspective) we just decide blindly how many layers and how many nodes without caring about what is what in the model? Is this the main difference?
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/p2t2/figaro/issues/589, or mute the threadhttps://github.com/notifications/unsubscribe/AFJkd6f9TN67PiYu2J-ie6HPW6zx1qZ1ks5qPN5ngaJpZM4I-Svl.
toncho11
Bayes nets and neural networks (ANN) have things in common - for example they are both directed graphical models.
What we are trying to estimate is the weights for each node that connects to the nodes of the next layer. So nodes can be represented as random variables whose distribution gives us the weights between the nodes in the ANN. Activation function must be set somewhere. This is how we define a model.
So the idea of training is to set the input and output as observed and then apply reasoning such as EMWithBP. Once we have these we can apply the model to new inputs (Observed) and infer (in top to bottom direction) to predict the output. The output is a random variable where we say >0.5 is class 1 and <0.5 is class 2.
Is the above correct? Is it similar to the Spam filter example? In the ANN perspective we just decide blindly how many layers and how many nodes without caring about what is what in the model? Is this the main difference?