search

emtiyaz / vadam

Code for ICML 2018 paper on "Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam" by Khan, Nielsen, Tangkaratt, Lin, Gal, and Srivastava
109 stars 25 forks source link

Log loss vs ELBO optimisation #5

Open skezle opened 5 years ago

skezle commented 5 years ago

In your UCI experiments, you set your objective function to a log loss (and likewise the BO objective). Surely one wants to perform VI with VADAM and should optimize an ELBO?

emtiyaz commented 5 years ago

I am not sure what the confusion is here, but if you read the paper, the whole point is to be able to write the gradients with respect to the loss of the neural networks. This way we don't have to change the computation graph. The algorithm still is able to perform VI which is the main contribution of the paper. I hope this makes sense.