search

bayesiains / nflows

Normalizing flows in PyTorch
MIT License
845 stars 118 forks source link

Problem with transforming between distributions #59

Closed rmastand closed 2 years ago

rmastand commented 2 years ago

I am trying to use nflows to map between two nontrivial probability distributions p_1, p_2. I first train a flow to learn distribution 1 with 

transform_1 = CompositeTransform(transforms_1)
flow_1 = Flow(transform_1, base_dist)

for an array of transform objects and a standard normal base_dist. After learning, when I sample from flow_1, the samples match the probability distribution as the data for p_1.

I then train a flow to map from p_1 to p_2 with

transform_2 = CompositeTransform(transforms_2)
flow_2 = Flow(transform_2, flow_1)

So I am using flow_1 as the base density for this transformation. After learning, when I sample from flow_2, the samples match the probability distribution as the data for p_2.

However, if I try to map from p_1 to p_2 by taking a sample from p_1 and running transformed_dat = flow_2.transform_to_noise(p_1), transformed_dat does not match the distribution from p_2.

Is there an additional step in transforming between datasets that I need to carry out?

imurray commented 2 years ago

If trained well, sampling flow_2 would draw "noise" from the base distribution p_1 and transform it to samples of p_2. You could reverse that sampling process with transform_to_noise to take samples from p_2 and turn them into samples from p_1. You'd need to pass p_1 samples through the "inverse" transform to get samples from p_2 (as the _sample method of the flow does).