karpathy / pytorch-normalizing-flows

Normalizing flows in PyTorch. Current intended use is education not production.
844 stars 98 forks source link

An important bug in flow.py #10

Open zhengqigao opened 1 year ago

zhengqigao commented 1 year ago

Take the forward function in the Affineflow class as an example. x0 is constructed by x[0], x[2], x[4],..., and x1 is constructed by x[1], x[3], x[5], ..... But z is obtained by directly concatenating z0 and z1 side by side: z = [z0[0], z0[1], ...., z0[N], z1[0], z1[1], ... z1[N]]. But the correct way should be: z = [z0[0], z1[0], z0[1], z1[1], z0[2], z1[2], ....z0[N], z1[N]].

The code currently has inconsistent output when calling forward and then backward.

x =torch.rand(10,4)
z, log_det = net(x) # here net is an Affineflow
x0, log_det2 = net(z)
# x0 and x will be different