api for tensorflow serving

[mazorigal]

Hi,

I am quite new to tensorflow and not sure if that the proper place to put my question. My main task is to support our data scientists in deploying tensorflow models into production properly rather then developing tensorflow models. One of the ideas for such deployment is to use tensorflow serving and then write a simple scala service that would be the gate to interact with the tensorflow serving service. (the tensorflow serving and scala app could run in the same docker) I was reading those 2 blogs: http://fdahms.com/2017/03/05/tensorflow-serving-jvm-client/ https://blog.codecentric.de/en/2017/01/hello-grpc-scalapb/

and it seems that main component for scala to communicate with tensorflow serving is via gRPC. However that requires to first get the proto files from tensorflow lib and then generate java / scala code from those.

Do you think that tensorflow_scala could be a leveraged in order to add also scala client to communicate with tensorflow serving ?

Thanks,

[eaplatanios]

@mazorigal I'm sorry for the late response but I've been working towards a paper deadline and it's been super busy. Adding integration support with TensorFlow serving should be easy in my library. I already support reading/writing Proto files for both models and tensor values. I think we pretty much have all the necessary pieces and so it shouldn't be hard.

I haven't used TensorFlow serving at all and I'm thus not aware of all the steps required. I can look into it after my deadline (maybe during the Christmas break), but it may also be useful if you told me what the main steps are (i.e., how you would do it in Python), and I can direct you as to how to do it with my API. How does that sound?

[mazorigal]

thanks for the reply. In order to use tensorflow serving via scala, I had first to copy protbuf files from tensorflow and from tensorflow serving git to my project and then generate scala code from those protbuf files via scala plugin (sbt-protoc). Once having the nessasery generated scala code, using grpc protocol (with the help of scalapb plugin) I was able to crate the required components in order to send requests and receive responses from the tensorflow serving. My thoughts were that if you already have the protbuf files in your project as well, and if you would also add the tensorflow serving protbuf files with the scala generated code, then others could just use your project to have already all the required data structures in order to create tensorflow serving requests.

If you would like, I can try to create public repo with a demo of what I have done in order to communicate with tensorflow serving.

[eaplatanios]

@mazorigal Yeah that sounds pretty simple and I should be able to support that through my API. A demo project would be very helpful indeed. :)

[eaplatanios]

I can then go on from there and post updated in this issue.

[mazorigal]

ok, I will try to find some time for that, but not promise that it would happen by the end of 2017.

[ryanmickler]

any progress on this? i'd be willing to help.

[eaplatanios]

I haven’t heard anything from @mazorigal and so I think there is no progress yet. I’d be more than happy to support if you want to take this on @ryanmickler. :)