Open sakulh opened 3 years ago
I'am trying to use saved model in Tensorflow Serving but without success.
I exported model:
yolo = YOLOv4() yolo.config.parse_names("yolov4-data/coco.names") yolo.config.parse_cfg("yolov4-data/yolov4-tiny.cfg") yolo.make_model() yolo.load_weights( "./yolov4-data/trained/yolov4-tiny-18000-step.weights", weights_type="yolo", ) yolo.model.save("./yolov4-data/saved_model/1")
I run this model in Tensorflow Serving:
tfserving | 2021-03-02 10:18:10.583329: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:152] Running initialization op on SavedModel bundle at path: /notebooks/object-detection/yolov4-data/saved_model/1 tfserving | 2021-03-02 10:18:10.727702: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:333] SavedModel load for tags { serve }; Status: success: OK. Took 3277171 microseconds. tfserving | 2021-03-02 10:18:10.727917: I tensorflow_serving/core/loader_harness.cc:87] Successfully loaded servable version {name: object-detection-yolov4 version: 1} tfserving | 2021-03-02 10:18:10.810755: I tensorflow_serving/model_servers/server.cc:355] Running gRPC ModelServer at 0.0.0.0:8500 ... tfserving | [warn] getaddrinfo: address family for nodename not supported tfserving | [evhttp_server.cc : 238] NET_LOG: Entering the event loop ... tfserving | 2021-03-02 10:18:10.813386: I tensorflow_serving/model_servers/server.cc:375] Exporting HTTP/REST API at:localhost:8501 ...
I'am able to get metadata of the saved model:
curl http://tfserving:8501/v1/models/object-detection-yolov4/metadata { "model_spec":{ "name": "object-detection-yolov4", "signature_name": "", "version": "1" } , "metadata": {"signature_def": { "signature_def": { "serving_default": { "inputs": { "input_1": { "dtype": "DT_FLOAT", "tensor_shape": { "dim": [ { "size": "-1", "name": "" }, { "size": "416", "name": "" }, { "size": "416", "name": "" }, { "size": "3", "name": "" } ], "unknown_rank": false }, "name": "serving_default_input_1:0" } }, "outputs": { "output_1": { "dtype": "DT_FLOAT", "tensor_shape": { "dim": [ { "size": "-1", "name": "" }, { "size": "13", "name": "" }, { "size": "13", "name": "" }, { "size": "255", "name": "" } ], "unknown_rank": false }, "name": "StatefulPartitionedCall:0" }, "output_2": { "dtype": "DT_FLOAT", "tensor_shape": { "dim": [ { "size": "-1", "name": "" }, { "size": "26", "name": "" }, { "size": "26", "name": "" }, { "size": "255", "name": "" } ], "unknown_rank": false }, "name": "StatefulPartitionedCall:1" } }, "method_name": "tensorflow/serving/predict" }, "__saved_model_init_op": { "inputs": {}, "outputs": { "__saved_model_init_op": { "dtype": "DT_INVALID", "tensor_shape": { "dim": [], "unknown_rank": true }, "name": "NoOp" } }, "method_name": "" } } } } }
But I'am not able to send image by rest and get detected objects:
import numpy as np from PIL import Image import requests data = np.array(Image.open(TEST_IMG)) req = { "inputs": {"input_1": data.tolist()} } resp = requests.post(SERVER_URL, json=req) resp.raise_for_status() print('response.status_code: {}'.format(resp.status_code)) print('response.content: {}'.format(resp.content))
Result: HTTPError: 400 Client Error: Bad Request for url: http://tfserving:8501/v1/models/object-detection-yolov4/versions/1:predict
HTTPError: 400 Client Error: Bad Request for url: http://tfserving:8501/v1/models/object-detection-yolov4/versions/1:predict
Any recommendations? Thank you. Lukas
Your payload should be {"instances": [data.tolist()]}
{"instances": [data.tolist()]}
I'am trying to use saved model in Tensorflow Serving but without success.
I exported model:
I run this model in Tensorflow Serving:
I'am able to get metadata of the saved model:
But I'am not able to send image by rest and get detected objects:
Result:
HTTPError: 400 Client Error: Bad Request for url: http://tfserving:8501/v1/models/object-detection-yolov4/versions/1:predict
Any recommendations? Thank you. Lukas