search

tensorflow / serving

A flexible, high-performance serving system for machine learning models
https://www.tensorflow.org/serving
Apache License 2.0
6.19k stars 2.19k forks source link

Provide example for resnetv2 client #433

Closed gitmalong closed 7 years ago

gitmalong commented 7 years ago

I'd be happy to see an example client for resnet v2.

gitmalong commented 7 years ago

I'm piggy backing off inception saved model and inception client.

In inception_saved_model.py I replaced


    # Run inference.
    logits, _ = inception_model.inference(images, NUM_CLASSES + 1)

with


 # Run inference.
 logits, _ = inception.inception_resnet_v2(images, num_classes=NUM_CLASSES, is_training=False)

(I'm using TF slim)

In inception_client.py I replaced

    request.inputs['images'].CopyFrom(
        tf.contrib.util.make_tensor_proto(data, shape=[1]))

with


    request.inputs['images'].CopyFrom(
        tf.contrib.util.make_tensor_proto(data, dtype.tf.string))

The export works fine but running the client results in

Traceback (most recent call last):
  File "/home/ubuntu/serving/bazel-bin/tensorflow_serving/example/resnet_client.runfiles/tf_serving/tensorflow_serving/example/resnet_client.py", line 56, in <module>
    tf.app.run()
  File "/home/ubuntu/serving/bazel-bin/tensorflow_serving/example/resnet_client.runfiles/org_tensorflow/tensorflow/python/platform/app.py", line 44, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "/home/ubuntu/serving/bazel-bin/tensorflow_serving/example/resnet_client.runfiles/tf_serving/tensorflow_serving/example/resnet_client.py", line 51, in main
    result = stub.Predict(request, 10.0)  # 10 secs timeout
  File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 323, in __call__
    self._request_serializer, self._response_deserializer)
  File "/usr/local/lib/python2.7/dist-packages/grpc/beta/_client_adaptations.py", line 209, in _blocking_unary_unary
    raise _abortion_error(rpc_error_call)
grpc.framework.interfaces.face.face.AbortionError: AbortionError(code=StatusCode.INVALID_ARGUMENT, details="slice index 0 of dimension 0 out of bounds.
     [[Node: map/strided_slice = StridedSlice[Index=DT_INT32, T=DT_INT32, _output_shapes=[[]], begin_mask=0, ellipsis_mask=0, end_mask=0, new_axis_mask=0, shrink_axis_mask=1, _device="/job:localhost/replica:0/task:0/cpu:0"](map/Shape, map/strided_slice/stack, map/strided_slice/stack_1, map/strided_slice/stack_2)]]")
gitmalong commented 7 years ago

I was able to get it working when I defined my placeholder as seen here and not like in inception_saved_model.py But the prediction takes about 15sec. I'm recompiling with cpu optimizations now.

Update: Compiling with optimizations helped a lot. exec time 0.834159851074 sec.

litgh commented 7 years ago

I am working on this too, Can you show your executable code for me?