search

bayesiains / nflows

Normalizing flows in PyTorch
MIT License
845 stars 118 forks source link

RealNVP can't reconstruct samples #37

Closed csalcedo001 closed 3 years ago

csalcedo001 commented 3 years ago

Hi!

I was playing with RealNVP and tried to reconstruct samples after a forward and backward pass through the flow. After training the network on moons dataset, I ran the following program

x, label = datasets.make_moons(1000, noise=.05)
x = torch.from_numpy(x.astype(np.float32))

with torch.no_grad():
    z, _ = model._transform(x, context=torch.nn.Identity()(None))
    x_hat, _ = model._transform(z, context=torch.nn.Identity()(None))

The following plots show the distribution of the points after each pass

Original x: realnvp_x

Normalized z: realnvp_z

Reconstruction x_hat: realnvp_x_hat

The forward pass looks great, since it forms a normal distribution. However, the backward pass doesn't reconstruct the input x_hat similar to x as expected. Any feedback on what is happening would be of great help. Thanks!

csalcedo001 commented 3 years ago

Double-checking my code I just realized my program was performing two forward passes instead of a forward and backward pass. After changing the last line to x_hat, _ = model._transform.inverse(z, context=torch.nn.Identity()(None)) the reconstruction was perfect.

Sorry for the false alarm, I'm closing this issue.