Open Alaya-in-Matrix opened 5 years ago
Also, from the code of PBS, the log-likelihood is calculated like this:
test_ll = np.mean(-0.5 * np.log(2 * math.pi * (v + v_noise)) - \
0.5 * (y_test - m)**2 / (v + v_noise)
Which seems to differ from the way log likelihood is caluclated in the MC-dropout code.
I don't quite understand the calculation of the log-likelihood
why is the
logsumexp
used? and why are the predictive variances not used?I tried to calculate the test log likelihood like this:
And it usually generates slightly worse log likelihood. For example, using the
concrete
dataset, with split id set to 19, the log likelihood given by the original code is -3.17, while the log likelihood given by the above code is -3.25.