Why is model put in train mode in validation?

kumar-shridhar / PyTorch-BayesianCNN

Bayesian Convolutional Neural Network with Variational Inference based on Bayes by Backprop in PyTorch.

MIT License

1.43k stars 324 forks source link

Why is model put in train mode in validation? #64

Closed zanklanecek closed 3 years ago

zanklanecek commented 3 years ago

https://github.com/kumar-shridhar/PyTorch-BayesianCNN/blob/d93bad543c3226cd0fe05c0cb0ba033c41b3caa6/main_bayesian.py#L67

Am I missing something or is this specific for Bayesian models? Thanks for your contribution with this code btw, appreciate it.

Piyush-555 commented 3 years ago

The model is constructed in such a way that weights are sampled if it is in training mode, and if in eval mode, mean values are taken as weights themselves, here. Since we wanted to infer from an ensemble, we keep the model in training mode.
Alternatively, you can simply eliminate the ensemble for loop and infer based on mean weights. That should also work fine.

zanklanecek commented 3 years ago

Thank you for explanation 👍