piEsposito / blitz-bayesian-deep-learning

A simple and extensible library to create Bayesian Neural Network layers on PyTorch.
GNU General Public License v3.0
918 stars 107 forks source link

Questions about prior distributions #106

Open closeyourmoise opened 2 years ago

closeyourmoise commented 2 years ago

Can this library only use Gaussian distribution as prior distribution? Are there other prior distributions? Another question is that if the input data and the prior distribution are Gaussian distribution, is the output of the network a Gaussian distribution?

GISCheng commented 2 years ago

I have a similar question. I trained a neural network for a regression task with several Bayesian layers. But the generated "CI acc" by "evaluate_regression" is really low. I guess it is because the original or predicted Y does not fit a Gaussian distribution.

cappelletto commented 1 year ago

I have been using this library for regressions tasks where the target is deviated from a Gaussian distribution. By doing target (Y) distribution shaping (e.g. logNormal, Box-Cox, Gamma fit) you can improve your results, both in terms of convergence rate during training and error in the predictions for the validation set. The easiest way to do it is by applying the transformation during the pre-processing. Of course, you will need to revert it once you want your data back into its original space