search

PaddlePaddle / Serving

A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）
Apache License 2.0
898 stars 250 forks source link

paddle_serving_server_gpu 部署时的错误。如运行错误 libcudart.so.10.1: cannot open shared object file: No such file or directory #1011

Closed GreenValue closed 3 years ago

GreenValue commented 3 years ago

https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.0-beta/deploy/serving

python -m paddle_serving_server_gpu.serve --model serving_server --port 9393 --gpu_ids 0

输出

mkdir: cannot create directory ‘workdir_0’: File exists
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
Going to Run Comand
/usr/local/python2.7.15/lib/python2.7/site-packages/paddle_serving_server_gpu/serving-gpu-trt-0.4.0/serving -enable_model_toolkit -inferservice_path workdir_0 -inferservice_file infer_service.prototxt -max_concurrency 0 -num_threads 2 -port 9393 -reload_interval_s 10 -resource_path workdir_0 -resource_file resource.prototxt -workflow_path workdir_0 -workflow_file workflow.prototxt -bthread_concurrency 2 -gpuid 0 -max_body_size 536870912 
/usr/local/python2.7.15/lib/python2.7/site-packages/paddle_serving_server_gpu/serving-gpu-trt-0.4.0/serving: error while loading shared libraries: libcudart.so.10.1: cannot open shared object file: No such file or directory

/usr/local/cuda-10.0/targets/x86_64-linux/lib/libcudart.so /usr/local/cuda-10.0/targets/x86_64-linux/lib/libcudart.so.10.0 /usr/local/cuda-10.0/targets/x86_64-linux/lib/libcudart.so.10.0.130

github-actions[bot] commented 3 years ago

Message that will be displayed on users' first issue

TeslaZhao commented 3 years ago

whl包安装的是哪个版本？

GreenValue commented 3 years ago

pip install paddle-serving-client -i https://mirror.baidu.com/pypi/simple

Successfully installed grpcio-1.33.2 grpcio-tools-1.33.2 paddle-serving-client-0.4.0

pip install paddle-serving-server-gpu -i https://mirror.baidu.com/pypi/simple

Successfully installed func-timeout-4.3.5 paddle-serving-server-gpu-0.4.0.100
TeslaZhao commented 3 years ago

需要加上版本号，使用 -U 参数重新安装一遍

pip install paddle-serving-client==0.4.0 pip install paddle-serving-server==0.4.0 # CPU pip install paddle-serving-app==0.2.0 pip install paddle-serving-server-gpu==0.4.0.post9 # GPU with CUDA9.0 pip install paddle-serving-server-gpu==0.4.0.post10 # GPU with CUDA10.0 pip install paddle-serving-server-gpu==0.4.0.100 # GPU with CUDA10.1+TensorRT

GreenValue commented 3 years ago

pip install paddle-serving-server-gpu==0.4.0.post10运行后，paddle_serving_server_gpu 启动OK。

但测试部署的服务时， python ../../deploy/serving/test_client.py ../../demo/road554.png 输出错误：

Traceback (most recent call last):
  File "../../deploy/serving/test_client.py", line 18, in <module>
    from paddle_serving_app.reader import *
ImportError: No module named paddle_serving_app.reader

paddle_serving_app.reader，如何安装？

GreenValue commented 3 years ago

用-U重新安装

pip install -U paddle-serving-client==0.4.0
pip install -U paddle-serving-server==0.4.0 # CPU
pip install -U paddle-serving-app==0.2.0
pip install -U paddle-serving-server-gpu==0.4.0.post10 # GPU with CUDA10.0

执行： python ../../deploy/serving/test_client.py ../../demo/road554.png

输出错误 No module named shapely.geometry：

Traceback (most recent call last):
  File "../../deploy/serving/test_client.py", line 18, in <module>
    from paddle_serving_app.reader import *
  File "/usr/local/python2.7.15/lib/python2.7/site-packages/paddle_serving_app/reader/__init__.py", line 15, in <module>
    from .image_reader import ImageReader, File2Image, URL2Image, Sequential, Normalize, Base64ToImage
  File "/usr/local/python2.7.15/lib/python2.7/site-packages/paddle_serving_app/reader/image_reader.py", line 24, in <module>
    from shapely.geometry import Polygon
ImportError: No module named shapely.geometry
TeslaZhao commented 3 years ago

方法1： pip install shapely==1.7.0

方法2： pip install -r python/requirements.txt

GreenValue commented 3 years ago

执行： pip install -r requirements.txt

输出错误：

/usr/local/python2.7.15/lib/python2.7/site-packages/OpenSSL/crypto.py:12: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.
  from cryptography import x509
Ignoring typeguard: markers 'python_version >= "3.4"' don't match your environment
Requirement already satisfied: tqdm in /usr/local/python2.7.15/lib/python2.7/site-packages (from -r requirements.txt (line 1)) (4.56.0)
Collecting visualdl>=2.0.0b (from -r requirements.txt (line 3))
  Could not find a version that satisfies the requirement visualdl>=2.0.0b (from -r requirements.txt (line 3)) (from versions: 0.0.1a0, 0.0.1a1, 0.0.1a2, 0.0.2, 1.0.0, 1.1.0, 1.2.0, 1.2.1, 1.3.0)
No matching distribution found for visualdl>=2.0.0b (from -r requirements.txt (line 3))
GreenValue commented 3 years ago

安装PaddleDetection的太新了？ git clone https://gitee.com/paddlepaddle/PaddleDetection

GreenValue commented 3 years ago

requirements.txt中visualdl版本我改成1.3.0，就好了。 cat requirements.txt

tqdm
typeguard ; python_version >= '3.4'
visualdl>=1.3.0
opencv-python
PyYAML
shapely
TeslaZhao commented 3 years ago

Serving python版本支持2.7，3.5，3.6，3.7，3.8

GreenValue commented 3 years ago 
python -V
Python 2.7.15

谢谢。应该是我requirements没有安装成功导致的。 pip install -r requirements.txt

TeslaZhao commented 3 years ago 
python -V
Python 2.7.15

谢谢。应该是我requirements没有安装成功导致的。 pip install -r requirements.txt

好的

GreenValue commented 3 years ago

测试代码 test_client.py 会自动创建output文件夹，并在output下生成bbox.json和road554.png两个文件

但output没有任何输出！

root@ubuntu:/paddle/PaddleDetection/PaddleDetection/inference_model/yolov3_mobilenet_v1_roadsign# python ../../deploy/serving/test_client.py ../../demo/road554.png

WARNING: Logging before InitGoogleLogging() is written to STDERR
I0202 03:47:34.700326 30185 naming_service_thread.cpp:209] brpc::policy::ListNamingService("127.0.0.1:9393"): added 1
feed_var {
  name: "image"
  alias_name: "image"
  is_lod_tensor: false
  feed_type: 1
  shape: 3
  shape: 608
  shape: 608
}
feed_var {
  name: "im_size"
  alias_name: "im_size"
  is_lod_tensor: false
  feed_type: 2
  shape: 2
}
fetch_var {
  name: "multiclass_nms_0.tmp_0"
  alias_name: "multiclass_nms_0.tmp_0"
  is_lod_tensor: true
  fetch_type: 1
  shape: -1
}
GreenValue commented 3 years ago

用Paddle Serving的波士顿房价预测的例子，结果OK。说明Paddle Serving本身没有问题吧。

WARNING: Logging before InitGoogleLogging() is written to STDERR
I0202 10:10:28.588372 32360 naming_service_thread.cpp:209] brpc::policy::ListNamingService("127.0.0.1:9292"): added 1
{'price': array([[18.901152]], dtype=float32)} // price正常输出

root@ubuntu: /paddle/test_serving/housing python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9292

TeslaZhao commented 3 years ago

使用paddle Detection套件启动预测，没有结果吗？

GreenValue commented 3 years ago

是的。 paddle Detection套件启动预测没有结果。

测试代码 test_client.py 会自动创建output文件夹，并在output下生成bbox.json和road554.png两个文件

但output没有任何输出！

  • 命令

root@ubuntu:/paddle/PaddleDetection/PaddleDetection/inference_model/yolov3_mobilenet_v1_roadsign# python ../../deploy/serving/test_client.py ../../demo/road554.png

WARNING: Logging before InitGoogleLogging() is written to STDERR
I0202 03:47:34.700326 30185 naming_service_thread.cpp:209] brpc::policy::ListNamingService("127.0.0.1:9393"): added 1
  • cat serving_client/serving_client_conf.prototxt
feed_var {
  name: "image"
  alias_name: "image"
  is_lod_tensor: false
  feed_type: 1
  shape: 3
  shape: 608
  shape: 608
}
feed_var {
  name: "im_size"
  alias_name: "im_size"
  is_lod_tensor: false
  feed_type: 2
  shape: 2
}
fetch_var {
  name: "multiclass_nms_0.tmp_0"
  alias_name: "multiclass_nms_0.tmp_0"
  is_lod_tensor: true
  fetch_type: 1
  shape: -1
}
  • paddle_serving_server_gpu.serve已启动 root@ubuntu:python -m paddle_serving_server_gpu.serve --model serving_server --port 9393 --gpu_ids 0
mkdir: cannot create directory ‘workdir_0’: File exists
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
Going to Run Comand
/usr/local/python2.7.15/lib/python2.7/site-packages/paddle_serving_server_gpu/serving-gpu-cuda10-0.4.0/serving -enable_model_toolkit -inferservice_path workdir_0 -inferservice_file infer_service.prototxt -max_concurrency 0 -num_threads 2 -port 9393 -reload_interval_s 10 -resource_path workdir_0 -resource_file resource.prototxt -workflow_path workdir_0 -workflow_file workflow.prototxt -bthread_concurrency 2 -gpuid 0 -max_body_size 536870912 
I0100 00:00:00.000000 30087 op_repository.h:68] RAW: Succ regist op: GeneralCopyOp
I0100 00:00:00.000000 30087 op_repository.h:68] RAW: Succ regist op: GeneralReaderOp
I0100 00:00:00.000000 30087 op_repository.h:68] RAW: Succ regist op: GeneralInferOp
I0100 00:00:00.000000 30087 op_repository.h:68] RAW: Succ regist op: GeneralDistKVInferOp
I0100 00:00:00.000000 30087 op_repository.h:68] RAW: Succ regist op: GeneralResponseOp
I0100 00:00:00.000000 30087 op_repository.h:68] RAW: Succ regist op: GeneralTextResponseOp
I0100 00:00:00.000000 30087 op_repository.h:68] RAW: Succ regist op: GeneralDistKVQuantInferOp
I0100 00:00:00.000000 30087 op_repository.h:68] RAW: Succ regist op: GeneralTextReaderOp
I0100 00:00:00.000000 30087 service_manager.h:79] RAW: Service[LoadGeneralModelService] insert successfully!
I0100 00:00:00.000000 30087 load_general_model_service.pb.h:333] RAW: Success regist service[LoadGeneralModelService][PN5baidu14paddle_serving9predictor26load_general_model_service27LoadGeneralModelServiceImplE]
I0100 00:00:00.000000 30087 service_manager.h:79] RAW: Service[GeneralModelService] insert successfully!
I0100 00:00:00.000000 30087 general_model_service.pb.h:1507] RAW: Success regist service[GeneralModelService][PN5baidu14paddle_serving9predictor13general_model23GeneralModelServiceImplE]
I0100 00:00:00.000000 30087 factory.h:155] RAW: Succ insert one factory, tag: FLUID_GPU_ANALYSIS, base type N5baidu14paddle_serving9predictor11InferEngineE
W0100 00:00:00.000000 30087 fluid_gpu_engine.cpp:27] RAW: Succ regist factory: ::baidu::paddle_serving::predictor::FluidInferEngine<FluidGpuAnalysisCore>->::baidu::paddle_serving::predictor::InferEngine, tag: FLUID_GPU_ANALYSIS in macro!
I0100 00:00:00.000000 30087 factory.h:155] RAW: Succ insert one factory, tag: FLUID_GPU_ANALYSIS_DIR, base type N5baidu14paddle_serving9predictor11InferEngineE
W0100 00:00:00.000000 30087 fluid_gpu_engine.cpp:33] RAW: Succ regist factory: ::baidu::paddle_serving::predictor::FluidInferEngine< FluidGpuAnalysisDirCore>->::baidu::paddle_serving::predictor::InferEngine, tag: FLUID_GPU_ANALYSIS_DIR in macro!
I0100 00:00:00.000000 30087 factory.h:155] RAW: Succ insert one factory, tag: FLUID_GPU_ANALYSIS_DIR_SIGMOID, base type N5baidu14paddle_serving9predictor11InferEngineE
W0100 00:00:00.000000 30087 fluid_gpu_engine.cpp:39] RAW: Succ regist factory: ::baidu::paddle_serving::predictor::FluidInferEngine< FluidGpuAnalysisDirWithSigmoidCore>->::baidu::paddle_serving::predictor::InferEngine, tag: FLUID_GPU_ANALYSIS_DIR_SIGMOID in macro!
I0100 00:00:00.000000 30087 factory.h:155] RAW: Succ insert one factory, tag: FLUID_GPU_NATIVE, base type N5baidu14paddle_serving9predictor11InferEngineE
W0100 00:00:00.000000 30087 fluid_gpu_engine.cpp:44] RAW: Succ regist factory: ::baidu::paddle_serving::predictor::FluidInferEngine<FluidGpuNativeCore>->::baidu::paddle_serving::predictor::InferEngine, tag: FLUID_GPU_NATIVE in macro!
I0100 00:00:00.000000 30087 factory.h:155] RAW: Succ insert one factory, tag: FLUID_GPU_NATIVE_DIR, base type N5baidu14paddle_serving9predictor11InferEngineE
W0100 00:00:00.000000 30087 fluid_gpu_engine.cpp:49] RAW: Succ regist factory: ::baidu::paddle_serving::predictor::FluidInferEngine<FluidGpuNativeDirCore>->::baidu::paddle_serving::predictor::InferEngine, tag: FLUID_GPU_NATIVE_DIR in macro!
I0100 00:00:00.000000 30087 factory.h:155] RAW: Succ insert one factory, tag: FLUID_GPU_NATIVE_DIR_SIGMOID, base type N5baidu14paddle_serving9predictor11InferEngineE
W0100 00:00:00.000000 30087 fluid_gpu_engine.cpp:55] RAW: Succ regist factory: ::baidu::paddle_serving::predictor::FluidInferEngine< FluidGpuNativeDirWithSigmoidCore>->::baidu::paddle_serving::predictor::InferEngine, tag: FLUID_GPU_NATIVE_DIR_SIGMOID in macro!
I0100 00:00:00.000000 30087 factory.h:155] RAW: Succ insert one factory, tag: FLUID_CPU_ANALYSIS, base type N5baidu14paddle_serving9predictor11InferEngineE
W0100 00:00:00.000000 30087 fluid_cpu_engine.cpp:25] RAW: Succ regist factory: ::baidu::paddle_serving::predictor::FluidInferEngine<FluidCpuAnalysisCore>->::baidu::paddle_serving::predictor::InferEngine, tag: FLUID_CPU_ANALYSIS in macro!
I0100 00:00:00.000000 30087 factory.h:155] RAW: Succ insert one factory, tag: FLUID_CPU_ANALYSIS_DIR, base type N5baidu14paddle_serving9predictor11InferEngineE
W0100 00:00:00.000000 30087 fluid_cpu_engine.cpp:31] RAW: Succ regist factory: ::baidu::paddle_serving::predictor::FluidInferEngine< FluidCpuAnalysisDirCore>->::baidu::paddle_serving::predictor::InferEngine, tag: FLUID_CPU_ANALYSIS_DIR in macro!
I0100 00:00:00.000000 30087 factory.h:155] RAW: Succ insert one factory, tag: FLUID_CPU_ANALYSIS_DIR_SIGMOID, base type N5baidu14paddle_serving9predictor11InferEngineE
W0100 00:00:00.000000 30087 fluid_cpu_engine.cpp:37] RAW: Succ regist factory: ::baidu::paddle_serving::predictor::FluidInferEngine< FluidCpuAnalysisDirWithSigmoidCore>->::baidu::paddle_serving::predictor::InferEngine, tag: FLUID_CPU_ANALYSIS_DIR_SIGMOID in macro!
I0100 00:00:00.000000 30087 factory.h:155] RAW: Succ insert one factory, tag: FLUID_CPU_NATIVE, base type N5baidu14paddle_serving9predictor11InferEngineE
W0100 00:00:00.000000 30087 fluid_cpu_engine.cpp:42] RAW: Succ regist factory: ::baidu::paddle_serving::predictor::FluidInferEngine<FluidCpuNativeCore>->::baidu::paddle_serving::predictor::InferEngine, tag: FLUID_CPU_NATIVE in macro!
I0100 00:00:00.000000 30087 factory.h:155] RAW: Succ insert one factory, tag: FLUID_CPU_NATIVE_DIR, base type N5baidu14paddle_serving9predictor11InferEngineE
W0100 00:00:00.000000 30087 fluid_cpu_engine.cpp:47] RAW: Succ regist factory: ::baidu::paddle_serving::predictor::FluidInferEngine<FluidCpuNativeDirCore>->::baidu::paddle_serving::predictor::InferEngine, tag: FLUID_CPU_NATIVE_DIR in macro!
I0100 00:00:00.000000 30087 factory.h:155] RAW: Succ insert one factory, tag: FLUID_CPU_NATIVE_DIR_SIGMOID, base type N5baidu14paddle_serving9predictor11InferEngineE
W0100 00:00:00.000000 30087 fluid_cpu_engine.cpp:53] RAW: Succ regist factory: ::baidu::paddle_serving::predictor::FluidInferEngine< FluidCpuNativeDirWithSigmoidCore>->::baidu::paddle_serving::predictor::InferEngine, tag: FLUID_CPU_NATIVE_DIR_SIGMOID in macro!
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0202 03:46:11.940208 30087 analysis_predictor.cc:138] Profiler is deactivated, and no profiling report will be generated.
I0202 03:46:11.944368 30087 analysis_predictor.cc:875] MODEL VERSION: 1.8.5
I0202 03:46:11.944380 30087 analysis_predictor.cc:877] PREDICTOR VERSION: 1.8.4
I0202 03:46:11.944592 30087 analysis_predictor.cc:474] ir_optim is turned off, no IR pass will be executed
--- Running analysis [ir_graph_build_pass]
--- Running analysis [ir_graph_clean_pass]
--- Running analysis [ir_analysis_pass]
--- Running analysis [ir_params_sync_among_devices_pass]
I0202 03:46:11.989709 30087 ir_params_sync_among_devices_pass.cc:41] Sync params from CPU to GPU
--- Running analysis [adjust_cudnn_workspace_size_pass]
--- Running analysis [inference_op_replace_pass]
--- Running analysis [memory_optimize_pass]
I0202 03:46:12.040207 30087 memory_optimize_pass.cc:223] Cluster name : im_size  size: 8
I0202 03:46:12.040220 30087 memory_optimize_pass.cc:223] Cluster name : yolo_box0.tmp_0  size: 17328
I0202 03:46:12.040222 30087 memory_optimize_pass.cc:223] Cluster name : batch_norm_15.tmp_3  size: 2957312
I0202 03:46:12.040225 30087 memory_optimize_pass.cc:223] Cluster name : image  size: 4435968
I0202 03:46:12.040225 30087 memory_optimize_pass.cc:223] Cluster name : transpose_0.tmp_0  size: 17328
I0202 03:46:12.040227 30087 memory_optimize_pass.cc:223] Cluster name : batch_norm_9.tmp_2  size: 5914624
I0202 03:46:12.040228 30087 memory_optimize_pass.cc:223] Cluster name : batch_norm_8.tmp_3  size: 5914624
I0202 03:46:12.040230 30087 memory_optimize_pass.cc:223] Cluster name : batch_norm_2.tmp_2  size: 23658496
I0202 03:46:12.040232 30087 memory_optimize_pass.cc:223] Cluster name : yolo_box1.tmp_0  size: 69312
I0202 03:46:12.040235 30087 memory_optimize_pass.cc:223] Cluster name : conv2d_1.tmp_0  size: 23658496
--- Running analysis [ir_graph_to_program_pass]
I0202 03:46:12.052462 30087 analysis_predictor.cc:496] ======= optimize end =======
I0202 03:46:12.055174 30093 analysis_predictor.cc:138] Profiler is deactivated, and no profiling report will be generated.
I0202 03:46:12.056335 30094 analysis_predictor.cc:138] Profiler is deactivated, and no profiling report will be generated.
I0202 03:46:12.057348 30095 analysis_predictor.cc:138] Profiler is deactivated, and no profiling report will be generated.
W0202 03:46:21.838639 30095 device_context.cc:252] Please NOTE: device: 0, CUDA Capability: 75, Driver API Version: 11.0, Runtime API Version: 10.0
W0202 03:46:21.840962 30095 device_context.cc:260] device: 0, cuDNN Version: 7.6.
TeslaZhao commented 3 years ago

明天我试一下哈

GreenValue commented 3 years ago

root@ubuntu:/paddle/PaddleDetection/PaddleDetection/inference_model/yolov3_mobilenet_v1_roadsign# python ../../deploy/serving/test_client.py ../../demo/road554.png

有时会报这个错：

WARNING: Logging before InitGoogleLogging() is written to STDERR
I0202 10:21:51.394723 32604 naming_service_thread.cpp:209] brpc::policy::ListNamingService("127.0.0.1:9393"): added 1
W0202 10:21:51.444041 32604 recorder.h:249] Input=-2147483648 to `general_model_baidu.paddle_serving.predictor.general_model.GeneralModelService_default_tag_139964696571984_infer_sync_ltc' underflows
W0202 10:21:51.444056 32604 percentile.cpp:119] Input=-6600112305 to `general_model_baidu.paddle_serving.predictor.general_model.GeneralModelService_default_tag_139964696571984_infer_sync_ltc' is negative, drop
GreenValue commented 3 years ago

root@ubuntu:/paddle/PaddleDetection/PaddleDetection/inference_model/yolov3_mobilenet_v1_roadsign# cat ../../deploy/serving/test_client.py test_client.py内容如下：

# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import sys
import numpy as np
from paddle_serving_client import Client
from paddle_serving_app.reader import *
import cv2
preprocess = Sequential([
    File2Image(), BGR2RGB(), Resize(
        (608, 608), interpolation=cv2.INTER_LINEAR), Div(255.0), Transpose(
            (2, 0, 1))
])

postprocess = RCNNPostprocess("label_list.txt", "output", [608, 608])
client = Client()

client.load_client_config("serving_client/serving_client_conf.prototxt")
client.connect(['127.0.0.1:9393'])
GreenValue commented 3 years ago

@TeslaZhao 如何了？谢谢

GreenValue commented 3 years ago

按照这里操作，有结果输出，OK. https://github.com/PaddlePaddle/Serving/tree/v0.4.0/python/examples/yolov4

上面有问题，估计是哪里配置有问题吧！