Collaborate?

[cfis]

Hi Andrew - thanks for starting this project. Over the last week a fair bit of time adding additional functionality and have done some rearragement of the code. I've particularly spent time on fleshing out the dataset object and updating the operation calls to use operation def info (that removes having to hard code special cases like TensorSliceDataset). My change are at (in the cfis branch):

https://github.com/cfis/tensorflow/tree/cfis

Let me know if and you you'd like to merge these changes.

Next up on my todo list was to add basic function support so that MapDataset can be implemented (manual function definition, not something like Autograph). Then after that, if we want to do training in ruby, looks like all the automatic differentiation code needs to be ported from python since its not in the c library (or taken from tensor-stream gem?)

Anyway, would love to work together on this.

[ankane]

Hey @cfis, thanks for offering to help - that'd be great.

TensorFlow is a huge library, so for the first milestone, I'd like to focus on the two quickstart tutorials with eager tensors. #1

Right now, they're blocked on automatic differentiation. I'm having a hard time following the C++ code the Python library calls and am waiting to hear back from a Google rep about if there's a way to call ComputeGradients via the C API (will report back when I do). To sum it up, we need to get this code working in Ruby. If that's something you'd like to help with, that'd be great.

After that, I think it'd make sense to continue adding tutorials. Will give feedback directly on the PR.

[cfis]

Basically I have the same thoughts, but I was starting with older tutorials that use the lower level apis versus Keras (just so I could learn them first). But of course you run straight into to the function issue and automatic differentiation issue.

I was going to deal with the function issue first (which automatic differentiation is going to need at some point anyway) - so I spent a couple days figuring out how it works. But I don't mind dealing with automatic differentiation first instead.

So I assume you see the tape.h/tape.cc code in the c library folder (which is weird since its C++ header file and not c). And then there is the the python library wrapping code in pywrap_tensorflow_internal.cc.

My first thought is basically do what python did. Create a ruby c extension that just copies the python code, and links to the tape.h header (although compiling against a C++ header file isn't going to work across compilers so that might become a pain). And if you go that far, see if the C++ api offers anything useful too (for example some of the gradient functions have been ported from Python, see tensorflow\tensorflow\cc\gradients). And then after all that, you still need to port the python code from tensorflow\tensorflow\python\eager.

Another approach is just port all the code to Ruby to start with and later move it back to C/C++. But I'd rather reuse what is there versus reivent something.

[ankane]

I think it's better to use Ruby + FFI so we don't have to deal with any C++ code and it'll work with JRuby. However, it's not clear to me how to do that with this example at the moment.

What do you mean by function issue?

[cfis]

Just took a look - unfortunately the tape.h interface is not exported from the tensorflow shared library. So its inaccessible to FFI (or for that matter a ruby c extension). The only way to access it would be having tensforflow installed and built (so you have access to the internal library files). That's kind of a show stopper since building tensorflow takes a couple hours (at least on my laptop) Bummer. Means all that code needs to be reimplemented (unless google surfaces an api for it).

On the example you show, you'd have to:

• Reimplement c/tape.cc
• Reimplement some of python\eager\pywrap_tfe_src.cc
• Reimplement some (all?) of python\eager*.py

So a daunting proposition.

On the functions, its important with eager execution. See:

https://github.com/tensorflow/community/blob/master/rfcs/20180918-functions-not-sessions-20.md https://www.tensorflow.org/guide/function

The idea is convert a local python function to a graph function so it can be optimized and/or executed on a computer besides your own. Functions are important for forward/back prop and required for apis like the mapdataset which take a function that is used to, well, map the data.

[ankane]

Re functions: wasn't sure what the "issue" part was referring to.

I'm not totally sure I understand what you're proposing for differentiation, but feel free to put together a proof of concept if you have an idea for it.

[cfis]

On differentiation - I'm just saying tensforflow implements it with a lot of Python code and some backend supporting C++ code that isn't accessible via FFI. So you'd have to rewrite all of it to ruby (or I guess to C/C++). Not a small task.

Note some of it has been previously ported though to ruby - see https://github.com/jedld/tensor_stream.

I don't really know how long it would take, but I'd think its a fairly significant effort working through what the existing tensorflow code is doing and then port it.