search

NVlabs / LSGM

The Official PyTorch Implementation of "LSGM: Score-based Generative Modeling in Latent Space" (NeurIPS 2021)
Other
340 stars 49 forks source link

Excellent Figure 1! #12

Closed judiebig closed 10 months ago

judiebig commented 1 year ago

Hi! I am so impressed by Figure 1 in your paper and would like to create a similar figure for my own paper. However, despite several attempts, I have been unable to replicate it. Would you mind providing some guidance or hints on how to draw this figure, particularly the background of the SDE trajectories? Thank you so much for your time and assistance!

GuHuangAI commented 10 months ago

@judiebig Hello, have you solved the problem?

judiebig commented 10 months ago

@judiebig Hello, have you solved the problem?

No I still have this question. I am still looking for potential solutions.

karstenkreis commented 10 months ago

The background of the SDE trajectories corresponds to the marginal probability of the diffused data distribution. Since the data distribution is a mixture of 3 Gaussians, calculating this is tractable: The marginal probabilities along the diffusion can be calculated by diffusing each of the 3 Gaussians (this is done just like in regular diffusion models, where we diffuse individual data points during training and calculate the diffused probabilities to use in the score matching objective, just that now we start from the means of the three Gaussians and we already have initial variances at the beginning of the diffusion corresponding to the variances of the initial Gaussians; more formally, solving the Fokker-Planck equation is doable analytically, if the initial distribution is Gaussian and the diffusion process has just affine coefficients, which is the case in this toy example). We can then add the resulting three probability densities up and normalize. All this is tractable, since we are only dealing with Gaussians. Now one can calculate the probability at any time t and position x along the diffusion process and plot a corresponding heat map using a fine discretization in t and x.

The parameters of the diffusion process used for creating the figure were adjusted to make the visualization look nice, and it can also help to scale the resulting marginal probability densities, this is, the heat maps, a bit to make the visualization look appealing. The green trajectories simply correspond to individual rolled out diffusion trajectories, which is essentially just Langevin dynamics. I used matplotlib for plotting, and the different parts of the full figure were combined in powerpoint.

judiebig commented 10 months ago

Thanks a lot!

GuHuangAI commented 10 months ago

Thanks a lot!

Hello, could you please provide some example codes?