Open Disciple7 opened 2 years ago
yghstill
commented
2 years ago @Disciple7
Disciple7
commented
2 years ago @Disciple7
- 确认下paddle serving的版本，看报错是nearest_interp_v2这个op不支持，可以咨询下paddle serving~
- 测试benchmark时，看报错是TRT_MIN_SUBGRAPH 不够，可以调整TRT_MIN_SUBGRAPH为40，然后重新导出模型试下：https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.4/ppdet/engine/export_utils.py#L31
paddleserving已经是最新版本了。 benchmark那里改了，也没有用，还是报同样的错误。
Disciple7
commented
2 years ago 补充一下，我的paddle是直接安装的预编译库https://paddle-inference-lib.bj.bcebos.com/2.2.2/python/Linux/GPU/x86-64_gcc8.2_avx_mkl_cuda10.1_cudnn7.6.5_trt6.0.1.5/paddlepaddle_gpu-2.2.2.post101-cp37-cp37m-linux_x86_64.whl 和这个有关系吗？
yghstill
commented
2 years ago @Disciple7 我们复现了这个问题，正在解决中。
yghstill
commented
2 years ago @Disciple7 经排查，需要使用Paddle inference 2.3rc或者develop版本，2.3rc的包即将发布，请先手动编译下使用吧
Disciple7
commented
2 years ago @Disciple7 经排查，需要使用Paddle inference 2.3rc或者develop版本，2.3rc的包即将发布，请先手动编译下使用吧
我装了paddlepaddle-gpu 2.3.0rc0的包，paddleserving还是报一样的no OpConverter错误，直接推理报错变成了CUDA runtime和CUDA Driver不匹配：
(paddle_env) ubuntu@public:~/lxd-storage/xzy/PaddleCV/PaddleDetection$ python deploy/python/infer.py --model_dir=./inference_model/yolov3_mobilenet_v1_270e_qat/yolov3_mobilenet_v1_qat --image_file=./dataset/fire_smoke_voc/images/00001.jpg --device=GPU --run_mode=trt_int8
W0428 13:08:26.092806 18784 init.cc:179] Compiled with WITH_GPU, but no GPU found in runtime.
/home/ubuntu/miniconda3/envs/paddle_env/lib/python3.7/site-packages/paddle/fluid/framework.py:478: UserWarning: You are using GPU version Paddle, but your CUDA device is not set properly. CPU device will be used by default.
"You are using GPU version Paddle, but your CUDA device is not set properly. CPU device will be used by default."
----------- Running Arguments -----------
batch_size: 1
camera_id: -1
cpu_threads: 1
device: GPU
enable_mkldnn: False
image_dir: None
image_file: ./dataset/fire_smoke_voc/images/00001.jpg
model_dir: ./inference_model/yolov3_mobilenet_v1_270e_qat/yolov3_mobilenet_v1_qat
output_dir: output
reid_batch_size: 50
reid_model_dir: None
run_benchmark: False
run_mode: trt_int8
save_images: False
save_mot_txt_per_img: False
save_mot_txts: False
scaled: False
threshold: 0.5
trt_calib_mode: False
trt_max_shape: 1280
trt_min_shape: 1
trt_opt_shape: 640
use_dark: True
use_gpu: False
video_file: None
------------------------------------------
----------- Model Configuration -----------
Model Arch: YOLO
Transform Order:
--transform op: Resize
--transform op: NormalizeImage
--transform op: Permute
--------------------------------------------
Traceback (most recent call last):
File "deploy/python/infer.py", line 773, in <module>
main()
File "deploy/python/infer.py", line 726, in main
enable_mkldnn=FLAGS.enable_mkldnn)
File "deploy/python/infer.py", line 94, in __init__
enable_mkldnn=enable_mkldnn)
File "deploy/python/infer.py", line 563, in load_predictor
predictor = create_predictor(config)
OSError: (External) CUDA error(35), CUDA driver version is insufficient for CUDA runtime version.
[Hint: 'cudaErrorInsufficientDriver'. This indicates that the installed NVIDIA CUDA driver is older than the CUDA runtime library. This is not a supported configuration.Users should install an updated NVIDIA display driver to allow the application to run.] (at /paddle/paddle/phi/backends/gpu/cuda/cuda_info.cc:66)但是我看nvidia-smi上面显示应该是匹配的，根据这个issue，CUDA 10.1对应的driver>=418.39即可：
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 430.64 Driver Version: 430.64 CUDA Version: 10.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce RTX 208... Off | 00000000:04:00.0 Off | N/A |
| 51% 57C P2 206W / 250W | 6423MiB / 11019MiB | 49% Default |
+-------------------------------+----------------------+----------------------+
| 1 GeForce RTX 208... Off | 00000000:83:00.0 Off | N/A |
| 27% 29C P8 15W / 250W | 10MiB / 11019MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
+-----------------------------------------------------------------------------+
PaddleDetection team appreciate any suggestion or problem you delivered~
Checklist:
描述问题/Describe the bug
按照官方教程，在VOC数据集上训练了一个yolov3_mobilenet_v1_270e_voc模型，量化config使用默认的yolov3_mobilenet_v1_qat.yml。导出为paddleserving模型后，不开启--use_trt选项时可以正常运行和检测，但无加速和压缩效果。开启--use_trt时报此错误。
在PaddleServing群里咨询管理后，管理称这是模型问题，建议联系PaddleDetection。
另外，用benchmark脚本测试时也同样报TensorRT相关错误，无法测试trt_fp32、trt_fp16和trt_int8，但use_cpu、use_gpu可以测试。
复现/Reproduction
您使用的命令是？/What command or script did you run? paddleserving命令：
benchmark测试命令：
您是否更改过代码或配置文件？您是否理解您所更改的内容？还请您提供所更改的部分代码。/Did you make any modifications on the code or config? Did you understand what you have modified? Please provide the codes that you modified.
否
VOC数据集
C++ Traceback (most recent call last):
Error Message Summary:
UnimplementedError: no OpConverter for optype [nearest_interp_v2] [Hint: it should not be null.] (at /paddle/paddle/fluid/inference/tensorrt/convert/op_converter.h:142)
Aborted (core dumped)
model_dir : ./inference_model/yolov3_mobilenet_v1_270e_qat/yolov3_mobilenet_v1_qat img_dir: demo/fire_smoke_demo model ./inference_model/yolov3_mobilenet_v1_270e_qat/yolov3_mobilenet_v1_qat, run_mode: trt_int8 ----------- Running Arguments ----------- batch_size: 1 camera_id: -1 cpu_threads: 1 device: GPU enable_mkldnn: False image_dir: demo/fire_smoke_demo image_file: None model_dir: ./inference_model/yolov3_mobilenet_v1_270e_qat/yolov3_mobilenet_v1_qat output_dir: output reid_batch_size: 50 reid_model_dir: None run_benchmark: True run_mode: trt_int8 save_images: False save_mot_txt_per_img: False save_mot_txts: False scaled: False threshold: 0.5 trt_calib_mode: False trt_max_shape: 1280 trt_min_shape: 1 trt_opt_shape: 640 use_dark: True use_gpu: False video_file: None
----------- Model Configuration ----------- Model Arch: YOLO Transform Order: --transform op: Resize --transform op: NormalizeImage --transform op: Permute
Traceback (most recent call last): File "deploy/python/infer.py", line 773, in
main()
File "deploy/python/infer.py", line 726, in main
enable_mkldnn=FLAGS.enable_mkldnn)
File "deploy/python/infer.py", line 94, in init
enable_mkldnn=enable_mkldnn)
File "deploy/python/infer.py", line 563, in load_predictor
predictor = create_predictor(config)
ValueError: (InvalidArgument) Pass tensorrt_subgraph_pass has not been registered. Please use the paddle inference library compiled with tensorrt or disable the tensorrt engine in inference configuration!
[Hint: Expected Has(pass_type) == true, but received Has(pass_type):0 != true:1.] (at /paddle/paddle/fluid/framework/ir/pass.h:236)