yang-song / score_sde_pytorch

PyTorch implementation for Score-Based Generative Modeling through Stochastic Differential Equations (ICLR 2021, Oral)
https://arxiv.org/abs/2011.13456
Apache License 2.0
1.66k stars 305 forks source link

Wrong Likelihood Computation for VE SDE #64

Open lujiazho opened 1 month ago

lujiazho commented 1 month ago

https://github.com/yang-song/score_sde_pytorch/blob/cb1f359f4aadf0ff9a5e122fe8fffc9451fd6e44/sde_lib.py#L244

I think this line should be

-N / 2. * np.log(2 * np.pi * self.sigma_max ** 2) - torch.sum((z-x) ** 2, dim=(1, 2, 3)) / (2 * self.sigma_max ** 2)

where the x is the initial x0 which should be passed into this function

instead of

-N / 2. * np.log(2 * np.pi * self.sigma_max ** 2) - torch.sum(z ** 2, dim=(1, 2, 3)) / (2 * self.sigma_max ** 2)

isn't it?

daihuiao commented 1 month ago

As I understand it, the prior probabilities are calculated here, and the final distribution is assumed to be a Gaussian distribution with a mean of 0, so x in (z-x) =0。 In addition, there is no x in the function parameter pass :)

lujiazho commented 1 month ago

As I understand it, the prior probabilities are calculated here, and the final distribution is assumed to be a Gaussian distribution with a mean of 0, so x in (z-x) =0。 In addition, there is no x in the function parameter pass :)

Thanks for your reply :) To my understanding, the prior is the p(x_T) which for VE SDE is as below image where the mean is actually x0 not 0