search

acids-ircam / pytorch_flows

Implementation and tutorials of normalizing flows with the novel distributions module
GNU General Public License v3.0
160 stars 17 forks source link

KL Formula #6

Closed Hakim777 closed 3 years ago

Hakim777 commented 3 years ago

Hi, Thanks for the tutorial.

I have a question. Why in the loss function you used the target distribution instead of the base distribution ? I remember that in the forward KL divergence there is no term for target distribution.

Thanks

esling commented 3 years ago

Hi,

This is exactly the point of generative flows that you can see in the literature : https://arxiv.org/abs/1807.03039

In this case what you do is to put your points inside the successive normalizing flows, and you regularize the final distribution of your flow to be a Gaussian (ie. put the same loss with an isotropic Gaussian density as density parameter). Then you can use the invertible property of flows to generate points from your learned density (sampling from the final Gaussian and using inverse transforms).

Best, Philippe