search

tobegit3hub / simple_tensorflow_serving

Generic and easy-to-use serving service for machine learning models
https://stfs.readthedocs.io
Apache License 2.0
757 stars 193 forks source link

Enhancements for 3rd framework integration #8

Closed leftnoteasy closed 6 years ago

leftnoteasy commented 6 years ago

@tobegit3hub, first of all, thanks for working on this framework, really appreciated!

While working on a prototype of YARN submarine project (YARN-8135). I saw this project and wanna to integrate this project into the model serving path.

There're a couple of requirements so far I can see: 1) Be able to read HDFS files. This is P0 requirement since model could be very large.

2) Allow recursively detect saved model path. Existing model serving expects folder with version numbers exist under the specified model path, it might be convenient for end users if the framework can recursively looking for models for serving. This is a P1 requirement.

3) Expose REST APIs to get models-signatures / configs / status, etc. Existing simple_tensorflow_serving only exposes POST APIs for model serving, it's better to expose GET APIs as well since simple_tensorflow_serving might be launched in a different machine. We can expose information like: a. Status of the service. (including model name, paths, statuses, etc.) b. Configs of the service. c. Model signatures. d. Generate clients examples (with proper endpoint).

Thanks again!

tobegit3hub commented 6 years ago

Thanks @leftnoteasy for the great suggestions.

We have considered supporting HDFS files and more RESTful APIs. We're really welcome to any feedback and looking forward to contributing this for Hadoop ecosystem 😃

tobegit3hub commented 6 years ago

Thanks to the help of community, most of these requested features are implemented.

  1. Support reading remote models in HDFS including Kerberos supported.
  2. Arbitrary recursively paths will not be supported because it may be ambiguous if we have multiple models in the same sub-directory. We want user to specify the complete model path just like TensorFlow Serving.
  3. Now we have more APIs for use to get model signature, generate example json and client in multiple programming languages.

I hope this helps and find an way to be part of the hadoop ecosystem. This issue may be closed and feel free to re-open if we have any other question.