Optimizing without known target density

[eublefar]

Hi

All tutorials I've seen regarding normalizing flows use loss with known target density, yours too, e.g.

def loss(density, zk, log_jacobians):
    sum_of_log_jacobians = sum(log_jacobians)
    return (-sum_of_log_jacobians - torch.log(density(zk) + 1e-9)).mean() # << density(zk)

How to optimize for data points instead? All that I can think of is just computing inverse of data sample into noise distribution and then optimizing log probability, is it correct way to approach it?

[esling]

Hi,

Indeed, this is exactly the point of generative flows that you can see in the literature : https://arxiv.org/abs/1807.03039

In this case what you do is to put your points inside the successive normalizing flows, and you regularize the final distribution of your flow to be a Gaussian (ie. put the same loss with an isotropic Gaussian density as density parameter). Then you can use the invertible property of flows to generate points from your learned density (sampling from the final Gaussian and using inverse transforms.

Best, Philippe

[eublefar]

Thank you for the explanation and patience!