search

emacski / tensorflow-serving-arm

TensorFlow Serving ARM - A project for cross-compiling TensorFlow Serving targeting popular ARM cores
Apache License 2.0
100 stars 16 forks source link

How to get tensorflow-serving-api #10

Closed patthongtra closed 3 years ago

patthongtra commented 3 years ago

Hi I am using the docker image emacski/tensorflow-serving:1.15.0. How can I get tensorflow-serving-api for my application (which sends requests to tensorflow-serving)? Thanks!

patthongtra commented 3 years ago

I found it - https://github.com/emacski/tensorflow-serving-arm-client :)

patthongtra commented 3 years ago

I found there is wheel version > 2.2.0 How to get version == 1.15.0 or which versions that work with tensorflow-serving version 1.15.0 ?

emacski commented 3 years ago

Hello,

Assuming you want to run a client on embeded arm, the short answer for versions < 2.2.0 is to use the REST API.

The official example using something like the python requests package:

import json
import requests

r = requests.post(
    "http://localhost:8501/v1/models/half_plus_two:predict",
    data=json.dumps({"instances":[1.0, 2.0, 5.0]}))

print(r.json())

Otherwise, you might find someone that has fully or partially extracted the grpc client from the tensorflow and tensorflow/serving projects for those earlier versions (github.com, pypi, etc.). Even if you do find something out there, keep in mind that installation success can be spotty based on dependency coverage for your platform and may require additional modification.

emacski commented 3 years ago

Haven't heard back