search

wjmaddox / drbayes

Code Repo for "Subspace Inference for Bayesian Deep Learning"
BSD 2-Clause "Simplified" License
79 stars 23 forks source link

Bug in SWAG implementation? #6

Closed Framartin closed 2 years ago

Framartin commented 2 years ago

Hello @wjmaddox,

Thanks for sharing your code. I found your paper very inspiring.

I wonder if the sampling of SWAG is correct. In particular, it appears that the standard Gaussian sample is multiplied by the vector of variances for the SWAG-Diagonal part:

https://github.com/wjmaddox/drbayes/blob/0efbd081b7ccecdb2fea8e949ad81065c26faa54/subspace_inference/posteriors/swag.py#L82

Whereas in the original SWAG paper and its implementation the standard Gaussian is multiplied by the vector of standard deviations to sample from N(0, diag(variances)):

https://github.com/wjmaddox/swa_gaussian/blob/b172d93278fdb92522c8fccb7c6bfdd6f710e4f0/swag/posteriors/swag.py#L121

Am I missing something? Thanks for your time.

wjmaddox commented 2 years ago

yes, that's a bug. sorry for the delay in responding. I'll update the code shortly.

Framartin commented 2 years ago

No worries! Thanks a lot for your answer and your time.