Looking for model server to support 2019.R2 models

[dtjensen]

It seems that the latest version (2019.1.1) only supports version "4" of the OpenVino models. Is there any guidance on how to host the most recent models (version=6) in the model server?

[mzegla]

Do you run model server in docker container or on bare host? If you run in docker, do you use image from Dockerhub or build your own? If you build your own, which Dockerfile do you use?

Model server shall support models supported by OpenVINO version it is run on. We observe no compatibility issues with R2 version, so if you want to run model server on bare host, you need to install OpenVINO R2 on it. If you want to run model server in docker container, use Dockerfile_binary_openvino to build your image (make sure you have R2 openvino toolkit tgz package in OpenVINO-model-server root directory). We are currently working on enabling R2 with other Dockerfiles.

[dtjensen]

I am trying to run in a Docker container. I was trying to use the pre-built docker image (2019.1.1) in Dockerhub but there were issues. Here is an example of what I saw:

2019-09-03 22:45:17,897 - ie_serving.models.model - ERROR - Error occurred while loading model version: {'xml_file': '/models/model1/1/person-detection-retail-0013.xml', 'bin_file': '/models/model1/1/person-detection-retail-0013.bin', 'mapping_config': None, 'version_number': 1, 'batch_size': None}
E0903 22:45:17.897165 140498535171840 model.py:253] Error occurred while loading model version: {'xml_file': '/models/model1/1/person-detection-retail-0013.xml', 'bin_file': '/models/model1/1/person-detection-retail-0013.bin', 'mapping_config': None, 'version_number': 1, 'batch_size': None}
2019-09-03 22:45:17,897 - ie_serving.models.model - ERROR - Content error: Error reading network: cannot parse future versions: 6
E0903 22:45:17.897570 140498535171840 model.py:254] Content error: Error reading network: cannot parse future versions: 6

Is that pre built images different than the output of the Dockerfile you mentioned?

[mzegla]

Well, we are working on upgrading docker image, Dockerfile and Dockerfile_intelpython to use OpenVINO R2. I suggest you trying building docker image with Dockerfile_binary_openvino and binary OpenVINO R2 toolkit package (for example by using make docker_build_bin command).

[dtjensen]

I have the server up and running using that Dockerfile.

Now, I just need some examples of how to invoke the Tensorflow predict API using the REST API. Are there any examples of that anywhere?

[mzegla]

From the logs you posted here I can see that you're using person-detection-retail-0013 model. https://docs.openvinotoolkit.org/2018_R5/_docs_Retail_object_detection_pedestrian_rmnet_ssd_0013_caffe_desc_person_detection_retail_0013.html Having more details about that model, we know it has single input named "input" with shape [1x3x320x544], so what you need to do is to convert your frame/image to numpy array in that shape. After that you can prepare JSON for your REST request in form {"inputs": <numpy_array>} and send it with post request to: <server_address>:<server_port>/v1/models/<model_name>/<model_version>:predict.

Our exemplary client was meant for classification models, especially ResNet-50, but maybe it would be helpful if you take a look at it: https://github.com/IntelAI/OpenVINO-model-server/blob/master/example_client/rest_serving_client.py

[dtrawins]

Support for OpenVINO 2019 R2 has been merged to master branch.