search

kumar-shridhar / PyTorch-BayesianCNN

Bayesian Convolutional Neural Network with Variational Inference based on Bayes by Backprop in PyTorch.
MIT License
1.42k stars 323 forks source link

Input argument for ELBO criterion #66

Open zanklanecek opened 3 years ago

zanklanecek commented 3 years ago

https://github.com/kumar-shridhar/PyTorch-BayesianCNN/blob/d93bad543c3226cd0fe05c0cb0ba033c41b3caa6/main_bayesian.py#L116

Your training set is divided on train and validation. Shouldn't here be (1-valid_size)*len(trainset)?

rahulrathnakumar commented 3 years ago

While I don't understand using the whole train set length for computing ELBO on a minibatch, what is the (1-valid_size) in your question? Thanks

Nebularaid2000 commented 2 years ago

I think it should be criterion = metrics.ELBO(batch_size).to(device) since the ELBO is calculated based on each mini-batch. Is it correct? @kumar-shridhar

zanklanecek commented 2 years ago

While I don't understand using the whole train set length for computing ELBO on a minibatch, what is the (1-valid_size) in your question? Thanks

I don't really understand the concept, but I am certain that they further divide their trainset into train and validation. So (1-valid_size) represents the actual size of the trainset.

If the actual length of training set must be used here, then my proposal should be correct. However, if ELBO is calculated based on each mini-batch, then @Nebularaid2000 solution should be the correct one.