Wider Ruby support using FFI

[headius]

I started playing with TensorFlow a bit today, and of course my first thought was doing it in Ruby.

Because I wanted something that would work with JRuby, my first path was to generate an FFI library. After some struggling with ffi_gen, I managed to generate this...and it worked!

https://gist.github.com/headius/446fc4980bf715bc5b358f1ba881aa80

I worked through some of the basic examples and, as you might expect, I saw an obvious need for better Ruby abstractions around the C API.

So I found tensorflow.rb!

Unfortunately tensorflow.rb uses a C extension. So I ask: why not replace that extension with the FFI code above?

[arafatkatze]

@headius First of all, Thanks a lot for your interest in tensorflow.rb.
I wish I had a better answer than this but currently I am extremely busy and caught up in my research work.
I haven't used FFI for my work, ever so I have no arguments against its usage. I tried using FFI for tensorflow.rb around a year ago but I let it go for some reason that. I understand how using FFI could be very useful both for JRuby and Ruby itself so I get why you asked this.

As for me using SWIG for working with Tensorflow C API, I would say that I have talked to a developers from Tensorflow team who worked with Tensorflow Bindings in Go and Java. They were very cool with me using SWIG as it worked very easily with the Tensorflow C API(Infact even Go used Go SWIG before moving to Go C for interacting with tensorflow C API) and later I built some abstractions over the SWIG work to make operations a lot easier than they were. Currently we have almost the same capabilities as the GO and JAVA API. But there is more to be done. How about this? I will try to look into this over the weekend and I will tell you if I find something cool.

Anyways, you are very welcome to look further. Thanks

[headius]

There's also an FFI generator from SWIG, which might make your changes less invasive...but have a look at my gist too. All the exported C functions, enums, etc are provided; as I mentioned I was even able to do some of the basic examples, albeit through considerably more verbose C than I used in Java and Python.

It may also be possible to maintain both backends until FFI can be proven out as a good solution going forward.

In any case, I may be playing with TensorFlow more in the next few weeks, and I'd like to do it in Ruby. If I can, I'll try to refactor what you have to work atop FFI.

[arafatkatze]

@headius That is really cool, Please go ahead.

[jtoy]

When we first started this project, SWIG was the sanctioned way to do a library. FFI is not the sanctioned method since about 6 months ago https://www.tensorflow.org/extend/language_bindings " Client languages should use the language's foreign function interface (FFI) to call into this C API to provide TensorFlow functionality." I would love to see this working with jruby!

[headius]

@jtoy That's wonderful! Their recognition of FFI means they'll probably be evolving the API with FFI in mind. Seems like it would be a good idea to try moving tensorflow.rb toward FFI. Luckily a lot of the work may be done already thanks to ffi_gen 😄

[jtoy]

@headius I would love to work on this, I dont have much experience with java/C integrations. That given, I would to help in other ways like put up some cash/bounty,testing, etc. I was actually looking into using java for writing a RNN recently, but didnt find anything suitable. And thanks for jruby, I've been using it more and more recently on some bigger projects :)

[headius]

I never got back to this but the ffi_gen or swig generator may still be an option. I will not have time to work on it, though, so I'm going to close this bug and hopefully someone else will be interested some day.