ynamic library libcublas.so.10.0
2019-12-11 13:50:47.813434: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2019-12-11 13:50:47.813444: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2019-12-11 13:50:47.813452: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2019-12-11 13:50:47.813461: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2019-12-11 13:50:47.813470: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2019-12-11 13:50:47.813510: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-12-11 13:50:47.813755: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-12-11 13:50:47.813972: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0
2019-12-11 13:50:47.813989: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-12-11 13:50:47.813994: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0
2019-12-11 13:50:47.813999: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N
2019-12-11 13:50:47.814049: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-12-11 13:50:47.814317: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-12-11 13:50:47.814543: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4319 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1660, pci bus id: 0000:01:00.0, compute capability: 7.5)
2019-12-11 13:50:47 INFO Put the model version: 1 online, path: ./models/deep_image_model/1
INFO:tensorflow:Restoring parameters from ./models/deep_image_model/1/variables/variables
2019-12-11 13:50:47 INFO Restoring parameters from ./models/deep_image_model/1/variables/variables
INFO:tensorflow:Restoring parameters from ./models/deep_image_model/1/variables/variables
2019-12-11 13:50:47 INFO Restoring parameters from ./models/deep_image_model/1/variables/variables
2019-12-11 13:50:47.947430: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-12-11 13:50:47.947698: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
name: GeForce GTX 1660 major: 7 minor: 5 memoryClockRate(GHz): 1.86
pciBusID: 0000:01:00.0
2019-12-11 13:50:47.947729: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2019-12-11 13:50:47.947740: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
2019-12-11 13:50:47.947751: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2019-12-11 13:50:47.947760: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2019-12-11 13:50:47.947770: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2019-12-11 13:50:47.947779: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2019-12-11 13:50:47.947790: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2019-12-11 13:50:47.947831: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-12-11 13:50:47.948080: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-12-11 13:50:47.948298: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0
2019-12-11 13:50:47.948315: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-12-11 13:50:47.948320: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0
2019-12-11 13:50:47.948325: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N
2019-12-11 13:50:47.948376: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-12-11 13:50:47.948624: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-12-11 13:50:47.948849: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4319 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1660, pci bus id: 0000:01:00.0, compute capability: 7.5)
2019-12-11 13:50:47 INFO Put the model version: 1 online, path: ./models/distributed_tensorflow_model/1
INFO:tensorflow:Restoring parameters from ./models/distributed_tensorflow_model/1/variables/variables
2019-12-11 13:50:47 INFO Restoring parameters from ./models/distributed_tensorflow_model/1/variables/variables
INFO:tensorflow:Restoring parameters from ./models/distributed_tensorflow_model/1/variables/variables
2019-12-11 13:50:47 INFO Restoring parameters from ./models/distributed_tensorflow_model/1/variables/variables
2019-12-11 13:50:48 INFO Try to get function from file: ./models/mxnet_mlp/preprocess_function.marshal
2019-12-11 13:50:48 INFO Try to get function from file: ./models/mxnet_mlp/postprocess_function.marshal
[13:50:48] src/nnvm/legacy_json_util.cc:209: Loading symbol saved by previous version v1.0.0. Attempting to upgrade...
[13:50:48] src/nnvm/legacy_json_util.cc:217: Symbol successfully upgraded!
2019-12-11 13:50:48 INFO Try to get function from file: ./models/onnx_mnist_model/preprocess_function.marshal
2019-12-11 13:50:48 INFO Try to get function from file: ./models/onnx_mnist_model/postprocess_function.marshal
2019-12-11 13:50:48 INFO Try to get function from file: ./models/scikitlearn_iris/preprocess_function.marshal
2019-12-11 13:50:48 ERROR Exception on / [GET]
Traceback (most recent call last):
File "/home/y/anaconda3/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
response = self.full_dispatch_request()
File "/home/y/anaconda3/lib/python3.7/site-packages/flask/app.py", line 1944, in full_dispatch_request
self.try_trigger_before_first_request_functions()
File "/home/y/anaconda3/lib/python3.7/site-packages/flask/app.py", line 1992, in try_trigger_before_first_request_functions
func()
File "/home/y/anaconda3/lib/python3.7/site-packages/simple_tensorflow_serving-0.8.1.1-py3.7.egg/simple_tensorflow_serving/./manager.py", line 70, in init
model_name, model_base_path)
File "/home/y/anaconda3/lib/python3.7/site-packages/simple_tensorflow_serving-0.8.1.1-py3.7.egg/simple_tensorflow_serving/./scikitlearn_inference_service.py", line 46, in init
self.model_base_path)
File "/home/y/anaconda3/lib/python3.7/site-packages/simple_tensorflow_serving-0.8.1.1-py3.7.egg/simple_tensorflow_serving/./preprocess_util.py", line 35, in get_preprocess_postprocess_function_from_model_path
preprocess_function = get_function_from_marshal_file(preprocess_file_path)
File "/home/y/anaconda3/lib/python3.7/site-packages/simple_tensorflow_serving-0.8.1.1-py3.7.egg/simple_tensorflow_serving/./preprocess_util.py", line 22, in get_function_from_marshal_file
loaded_function = marshal.loads(preprocess_function_string)
ValueError: bad marshal data (unknown type code)
2019-12-11 13:50:48 INFO 172.16.4.73 - - [11/Dec/2019 13:50:48] "GET / HTTP/1.1" 500 -
What I'v done:->
>cd simple_tensorflow_serving/simple_tensorflow_serving
>simple_tensorflow_serving --host=172.16.4.73 --model_config_file="../examples/model_config_file.json"
ynamic library libcublas.so.10.0 2019-12-11 13:50:47.813434: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0 2019-12-11 13:50:47.813444: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0 2019-12-11 13:50:47.813452: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0 2019-12-11 13:50:47.813461: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0 2019-12-11 13:50:47.813470: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7 2019-12-11 13:50:47.813510: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2019-12-11 13:50:47.813755: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2019-12-11 13:50:47.813972: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0 2019-12-11 13:50:47.813989: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix: 2019-12-11 13:50:47.813994: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0 2019-12-11 13:50:47.813999: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N 2019-12-11 13:50:47.814049: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2019-12-11 13:50:47.814317: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2019-12-11 13:50:47.814543: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4319 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1660, pci bus id: 0000:01:00.0, compute capability: 7.5) 2019-12-11 13:50:47 INFO Put the model version: 1 online, path: ./models/deep_image_model/1 INFO:tensorflow:Restoring parameters from ./models/deep_image_model/1/variables/variables 2019-12-11 13:50:47 INFO Restoring parameters from ./models/deep_image_model/1/variables/variables INFO:tensorflow:Restoring parameters from ./models/deep_image_model/1/variables/variables 2019-12-11 13:50:47 INFO Restoring parameters from ./models/deep_image_model/1/variables/variables 2019-12-11 13:50:47.947430: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2019-12-11 13:50:47.947698: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: name: GeForce GTX 1660 major: 7 minor: 5 memoryClockRate(GHz): 1.86 pciBusID: 0000:01:00.0 2019-12-11 13:50:47.947729: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0 2019-12-11 13:50:47.947740: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0 2019-12-11 13:50:47.947751: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0 2019-12-11 13:50:47.947760: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0 2019-12-11 13:50:47.947770: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0 2019-12-11 13:50:47.947779: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0 2019-12-11 13:50:47.947790: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7 2019-12-11 13:50:47.947831: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2019-12-11 13:50:47.948080: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2019-12-11 13:50:47.948298: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0 2019-12-11 13:50:47.948315: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix: 2019-12-11 13:50:47.948320: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0 2019-12-11 13:50:47.948325: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N 2019-12-11 13:50:47.948376: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2019-12-11 13:50:47.948624: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2019-12-11 13:50:47.948849: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4319 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1660, pci bus id: 0000:01:00.0, compute capability: 7.5) 2019-12-11 13:50:47 INFO Put the model version: 1 online, path: ./models/distributed_tensorflow_model/1 INFO:tensorflow:Restoring parameters from ./models/distributed_tensorflow_model/1/variables/variables 2019-12-11 13:50:47 INFO Restoring parameters from ./models/distributed_tensorflow_model/1/variables/variables INFO:tensorflow:Restoring parameters from ./models/distributed_tensorflow_model/1/variables/variables 2019-12-11 13:50:47 INFO Restoring parameters from ./models/distributed_tensorflow_model/1/variables/variables 2019-12-11 13:50:48 INFO Try to get function from file: ./models/mxnet_mlp/preprocess_function.marshal 2019-12-11 13:50:48 INFO Try to get function from file: ./models/mxnet_mlp/postprocess_function.marshal [13:50:48] src/nnvm/legacy_json_util.cc:209: Loading symbol saved by previous version v1.0.0. Attempting to upgrade... [13:50:48] src/nnvm/legacy_json_util.cc:217: Symbol successfully upgraded! 2019-12-11 13:50:48 INFO Try to get function from file: ./models/onnx_mnist_model/preprocess_function.marshal 2019-12-11 13:50:48 INFO Try to get function from file: ./models/onnx_mnist_model/postprocess_function.marshal 2019-12-11 13:50:48 INFO Try to get function from file: ./models/scikitlearn_iris/preprocess_function.marshal 2019-12-11 13:50:48 ERROR Exception on / [GET] Traceback (most recent call last): File "/home/y/anaconda3/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app response = self.full_dispatch_request() File "/home/y/anaconda3/lib/python3.7/site-packages/flask/app.py", line 1944, in full_dispatch_request self.try_trigger_before_first_request_functions() File "/home/y/anaconda3/lib/python3.7/site-packages/flask/app.py", line 1992, in try_trigger_before_first_request_functions func() File "/home/y/anaconda3/lib/python3.7/site-packages/simple_tensorflow_serving-0.8.1.1-py3.7.egg/simple_tensorflow_serving/./manager.py", line 70, in init model_name, model_base_path) File "/home/y/anaconda3/lib/python3.7/site-packages/simple_tensorflow_serving-0.8.1.1-py3.7.egg/simple_tensorflow_serving/./scikitlearn_inference_service.py", line 46, in init self.model_base_path) File "/home/y/anaconda3/lib/python3.7/site-packages/simple_tensorflow_serving-0.8.1.1-py3.7.egg/simple_tensorflow_serving/./preprocess_util.py", line 35, in get_preprocess_postprocess_function_from_model_path preprocess_function = get_function_from_marshal_file(preprocess_file_path) File "/home/y/anaconda3/lib/python3.7/site-packages/simple_tensorflow_serving-0.8.1.1-py3.7.egg/simple_tensorflow_serving/./preprocess_util.py", line 22, in get_function_from_marshal_file loaded_function = marshal.loads(preprocess_function_string) ValueError: bad marshal data (unknown type code) 2019-12-11 13:50:48 INFO 172.16.4.73 - - [11/Dec/2019 13:50:48] "GET / HTTP/1.1" 500 -
What I'v done:-> >cd simple_tensorflow_serving/simple_tensorflow_serving >simple_tensorflow_serving --host=172.16.4.73 --model_config_file="../examples/model_config_file.json"