question about the uncertainty part

[he-del]

Hi, I have read the part about the uncertainty quantification of neural networks, and I'm confused about the log-likelihood function you mentioned, i.e. -sum((y[:,1] - obs[:,1]).^2)/2σ^2 - sum(x.^2)/2σx^2, especially the value of σ and σw. Is there any reference for this part? Thanks! ​

[kailaix]

Hi, this log likelihood function arises from the Bayesian perspective of view. Consider inferring x given observations y, we have

p(x|y) ∝ p(y|x)p(x) => log(p(x|y)) = log p(y|x) + log p(x) + constant

• Assuming that the prior for x is Gaussian with zero mean and std σx, then log p(x) ∝ - sum(x.^2)/2σx^2
• Assuming that p(y|x) is Gassuain with mean obs and std σ, then log p(y|x) ∝ -sum((y[:,1] - obs[:,1]).^2)/2σ^2

You might want to look into some literature on Bayesian methods. Maybe this could be helpful: https://www.sagepub.com/sites/default/files/upm-binaries/18550_Chapter6.pdf

[he-del]

Thank you, this is helpful.