search

samjabrahams / tensorflow-white-paper-notes

Annotated notes and summaries of the TensorFlow white paper, along with SVG figures and links to documentation
MIT License
439 stars 61 forks source link

Graph model: cyclic or acyclic #6

Closed larissa95 closed 8 years ago

larissa95 commented 8 years ago

Hi, I read your book about TensorFlow. In there you wrote that the graph becomes acyclic by unenrolling the loops. Where have you found this information (in code,...)? How do you know how many cycles you have to unroll?

I found ambiguous information about that. For example this one is indicating that the graph might be cyclic: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/common_runtime/simple_placer.cc#L627

samjabrahams commented 8 years ago

Hi @larissa95: This question would be more appropriate for the book's repository, though it is good to note. TensorFlow does support cyclic graphs, through use of specific operations, such as tf.while_loop. In my experience teaching TensorFlow, it's been useful to teach people that the following "cyclic" code doesn't work the way a beginner might think it would:

a = tf.constant(3)
b = tf.mul(a, 2)
a = tf.add(b, 1)  # Does not "loop back" to previous variable `a`

To that end, I found it easy to simply say: "this doesn't work!", instead of getting muddled in the specifics of why one way doesn't work and how another function accomplishes it instead. I should be more careful with the wording, however (saying "TensorFlow does not support cyclic computation" is going to far).

As far as the "unrolling" section is concerned, that section is more about understanding what a cyclic computational graph means when you apply it to something like an RNN: the output from the previous "loop" feeds into the next iteration. It has less to do with TensorFlow-specific functionality.