search

abdulfatir / normalizing-flows

Understanding normalizing flows
MIT License
131 stars 20 forks source link

constraint for invertibility #1

Open chanshing opened 5 years ago

chanshing commented 5 years ago

Thanks for the nice work. May I know how and where do you enforce the constraint required to obtain invertible flows? Specifically, the condition w^Tu >= -1 as described in the appendix of the Normalizing flows paper.

abdulfatir commented 5 years ago

Thanks!

The following lines handle the condition. We construct u_hat that follows the condition required.

u_hat = (m(tf.tensordot(w, u, 2)) - tf.tensordot(w, u, 2)) * (w / tf.norm(w)) + u
affine = h_prime(tf.expand_dims(tf.reduce_sum(z_prev * w, -1), -1) + b) * w
sum_log_det_jacob += tf.log(eps + tf.abs(1 + tf.reduce_sum(affine * u_hat, -1)))
z_prev = z_prev + u_hat * h(tf.expand_dims(tf.reduce_sum(z_prev * w, -1), -1) + b)