search

eaplatanios / tensorflow_scala

TensorFlow API for the Scala Programming Language
http://platanios.org/tensorflow_scala/
Apache License 2.0
936 stars 96 forks source link

Recursive, Graph-based functions definitions #153

Open DirkToewe opened 5 years ago

DirkToewe commented 5 years ago

As @doofin pointed out in PR#143, the real power of TF comes into play when it's building blocks are combined using control flow. One of the most powerful means to that end are graph-based (recursive) function definitions.

Thanks to lazy values in Scala, recursive function could be quite elegantly defined in Tensorflow4Scala. It could look something along the lines of:

lazy val Fib = tf.func[Long,Long]{
  (n: Output[Long]) => tf.cond(
    n <= 1,
    1,
    Fib(n-1) + Fib(n-2)
  )
}

// ...

sess.run( FeedMap(), Seq(Fib(13)) )
eaplatanios commented 5 years ago

I don't this this is possible due to limitations in TensorFlow functions. In TF Scala, TF functions are effectively traced the first time they are invoked and given that they do not support recursion, something like the above example would unfortunately not work. I believe there are future plans to add support for recursion in TF functions, but not sure when that might happen.

DirkToewe commented 5 years ago

Are You sure? I believe it was at least possible at some point in the past, see for example this Stackoverflow Post. I believe I had recursion running at one point as well but it must have been many TF version's ago.

eaplatanios commented 5 years ago

Oh that's interesting. I hadn't actually looked into that before. I may take a look if I can find some time, but it is definitely not as simply as just defining a lazy Scala value, because we'd have to modify the function tracing code and add support for forward declarations of TF functions.

DirkToewe commented 5 years ago

Do You mean the implementation or the API? The implementation is not going to be trivial, I realize that. I will try to find out more myself as well.

The TF4S API in the end however could be as simple as the example above, I believe. And it would allow to write really clean and beautiful TF code in Scala once it does work.

eaplatanios commented 5 years ago

Yeah sorry...that's what I meant. I agree with respect to the API, but we would need to think about how to deal with forward declaration of recursive functions. I think there are some simple solutions, but the implementation will indeed not going to be trivial.

doofin commented 3 years ago

It seems that this is trival with the new tf eager ? I see there are eager ops in this project but haven't tested yet