The official code repository for the second edition of the O'Reilly book Generative Deep Learning: Teaching Machines to Paint, Write, Compose and Play.
In the book it says, page 183 "if training=True, we move forward (from data to latent space). If training=False, we move backwards through the layers (from latent space to data).
If training=True, then direction=-1.
If we check the for loop: for i in range(self.coupling_layers)[::direction]: then with a -1 instead of direction, we are moving backwards, not forward as the text say.
Let's focus on the training=False case because it is simpler, then direction is 1. Based on the previous for loop, we are moving forward (even though the text says that training=false means moving backwards).
With direction = 1, we get that the gate (gate = (direction - 1) / 2) is equals to 0.
If we substitute direction =1 and gate = 0 in:
(x * tf.exp(direction * s) + direction * t * tf.exp(gate * s))
we get x*tf.exp(s) + t. Which is what in page 178 it said it is the forward pass.
Going back to training=True, then direction = -1, then gate = -1. if we substitute these values in
(x * tf.exp(direction * s) + direction * t * tf.exp(gate * s))
we get xtf.exp(-s) -t tf.exp(-s) which is (z-t) * exp(-s) or the inverse pass.
So basically, if training is true, we are doing the inverse pass, from latent space to data. While if training is false, we are doing the forward pass, from data to latent space.
I have a printed copy of the book.
Page 178 says that the forward pass is z = x exp(s) + t. Page 179 says that the backwards/inverse pass is x =(z-t) exp(-s)
This matches some other guides such as this one
In the book it says, page 183 "if training=True, we move forward (from data to latent space). If training=False, we move backwards through the layers (from latent space to data).
If training=True, then direction=-1.
If we check the for loop:
for i in range(self.coupling_layers)[::direction]:then with a -1 instead of direction, we are moving backwards, not forward as the text say.Let's focus on the training=False case because it is simpler, then direction is 1. Based on the previous for loop, we are moving forward (even though the text says that training=false means moving backwards).
With direction = 1, we get that the gate (
gate = (direction - 1) / 2) is equals to 0.If we substitute direction =1 and gate = 0 in:
(x * tf.exp(direction * s) + direction * t * tf.exp(gate * s))we get x*tf.exp(s) + t. Which is what in page 178 it said it is the forward pass.
Going back to training=True, then direction = -1, then gate = -1. if we substitute these values in
(x * tf.exp(direction * s) + direction * t * tf.exp(gate * s))we get xtf.exp(-s) -t tf.exp(-s) which is (z-t) * exp(-s) or the inverse pass.
So basically, if training is true, we are doing the inverse pass, from latent space to data. While if training is false, we are doing the forward pass, from data to latent space.
What I am getting wrong?