search

yaringal / DropoutUncertaintyExps

Experiments used in "Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning"
Other
554 stars 122 forks source link

Calculation of Predictive Variance? #5

Open leimao opened 5 years ago

leimao commented 5 years ago

Hello Yarin,

It looks like that the description of the outputs in your predict method of the net class does not match to the actual output. https://github.com/yaringal/DropoutUncertaintyExps/blob/6eb4497628d12b0f300f4b4f6bdc386bebad565c/net/net.py#L95-L108

According to your publication, the predictive variance should be the sample variance of T stochastic forward passes plus the inverse model precision tau. (In your case, because the output y is a scalar, the variance are also scalars.) But it looks like that you did not add the inverse of tau when you are calculating the predictive "rmse". In addition, what is the estimate variance with additive noise?

Thank you very much.

Best,

Lei

malharjajoo commented 4 years ago

I agree with this (although I'm not sure about the last bit: why you are mentioning that variance is required for calculating predictive RMSE ....)

leimao commented 4 years ago

I agree with this (although I'm not sure about the last bit: why you are mentioning that variance is required for calculating predictive RMSE ....)

It has been half a year since I opened this. Now I almost totally forgot what I have written. I will come back to this later Orz.