I am currently doing a course project on optimizing ELBO with respect to variational (inference) model only.
For this following example,
https://github.com/blei-lab/edward/blob/master/examples/sigmoid_belief_network.py
I fixed the model parameters by adding trainable=Fasle in line 76 and 80. Then I printed out the negative ELBO, and found it increasing steadily for each iteration. However, when I fixed the variational parameters, the negative ELBO decreases as it is supposed to.
Is this normal since the ELBO and gradients are estimated? Any help will be appreciated!
Hello,
I am currently doing a course project on optimizing ELBO with respect to variational (inference) model only.
For this following example, https://github.com/blei-lab/edward/blob/master/examples/sigmoid_belief_network.py I fixed the model parameters by adding
trainable=Faslein line 76 and 80. Then I printed out the negative ELBO, and found it increasing steadily for each iteration. However, when I fixed the variational parameters, the negative ELBO decreases as it is supposed to.Is this normal since the ELBO and gradients are estimated? Any help will be appreciated!