Closed CharlesYXW closed 2 years ago
qingqing01
commented
3 years ago GPU上量化推理，需要设置run_mode为trt_int8 ，参考文档: https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.2/deploy/python/README.md#2-%E5%9F%BA%E4%BA%8Epython%E7%9A%84%E9%A2%84%E6%B5%8B
CharlesYXW
commented
3 years ago 使用命令：python deploy/python/infer.py --model_dir=output_inference/ppyolo_r50vd_qat_pact --device='GPU' --video_file=demo/1.mp4 --run_mode=trt_int8
进行对量化模型推理时依然报错：
----------- Running Arguments -----------
batch_size: 1
camera_id: -1
cpu_threads: 1
device: GPU
enable_mkldnn: False
image_dir: None
image_file: None
model_dir: output_inference/ppyolo_r50vd_qat_pact
output_dir: output
reid_batch_size: 50
reid_model_dir: None
run_benchmark: False
run_mode: trt_int8
save_images: False
save_mot_txts: False
threshold: 0.5
trt_calib_mode: False
trt_max_shape: 1280
trt_min_shape: 1
trt_opt_shape: 640
use_dark: True
use_gpu: False
video_file: demo/1.mp4
------------------------------------------
----------- Model Configuration -----------
Model Arch: YOLO
Transform Order:
--transform op: Resize
--transform op: NormalizeImage
--transform op: Permute
--------------------------------------------
Traceback (most recent call last):
File "deploy/python/infer.py", line 667, in <module>
main()
File "deploy/python/infer.py", line 596, in main
detector = Detector(
File "deploy/python/infer.py", line 78, in __init__
self.predictor, self.config = load_predictor(
File "deploy/python/infer.py", line 454, in load_predictor
predictor = create_predictor(config)
ValueError: (InvalidArgument) Pass tensorrt_subgraph_pass has not been registered. Please use the paddle inference library compiled with tensorrt or disable the tensorrt engine in inference configuration!
[Hint: Expected Has(pass_type) == true, but received Has(pass_type):0 != true:1.] (at /paddle/paddle/fluid/framework/ir/pass.h:216)看样子是说没用能编译TensorRT的推理库，我的Paddle版本2.1.3，cuda 11.0 ，TensorRT 7.2.3 ,在https://paddleinference.paddlepaddle.org.cn/user_guides/download_lib.html 中并没有找到cuda11.0能用的，请问如何解决
qingqing01
commented
3 years ago 
paddle-bot-old[bot]
commented
2 years ago Since this issue has not been updated for more than three months, it will be closed, if it is not solved or there is a follow-up one, please reopen it at any time and we will continue to follow up. It is recommended to pull and try the latest code first. 由于该问题超过三个月未更新，将会被关闭，若问题未解决或有后续问题，请随时重新打开（建议先拉取最新代码进行尝试），我们会继续跟进。
aileenfun
commented
2 years ago 你好，我也遇到了同样的问题。 平台是Win 10，rtx2060 现在我还没有做tensorRT联编，先尝试使用CPU进行推理，但是结果相同，没有报错，但是同样检测不到任何东西。
请问如果想用CPU进行部署的话，需要怎么调参数？
python tools/export_model.py -c configs/yolov3/yolov3_bamboo2.yml --output_dir=./inference_model -o weights=output/yolov3_bamboo2/best_model.pdparams
python deploy/python/infer.py --device=CPU --model_dir=./inference_model/yolov3_bamboo2 --image_file=./dataset/bamboo/testimg/1.jpg
Q:\paddletest\venv\lib\site-packages\paddle\vision\transforms\functional_pil.py:36: DeprecationWarning: NEAREST is deprecated and will be removed
in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
'nearest': Image.NEAREST,
Q:\paddletest\venv\lib\site-packages\paddle\vision\transforms\functional_pil.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed
in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
'bilinear': Image.BILINEAR,
Q:\paddletest\venv\lib\site-packages\paddle\vision\transforms\functional_pil.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed
in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
'bicubic': Image.BICUBIC,
Q:\paddletest\venv\lib\site-packages\paddle\vision\transforms\functional_pil.py:39: DeprecationWarning: BOX is deprecated and will be removed in P
illow 10 (2023-07-01). Use Resampling.BOX instead.
'box': Image.BOX,
Q:\paddletest\venv\lib\site-packages\paddle\vision\transforms\functional_pil.py:40: DeprecationWarning: LANCZOS is deprecated and will be removed
in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
'lanczos': Image.LANCZOS,
Q:\paddletest\venv\lib\site-packages\paddle\vision\transforms\functional_pil.py:41: DeprecationWarning: HAMMING is deprecated and will be removed
in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.
'hamming': Image.HAMMING
----------- Running Arguments -----------
action_file: None
batch_size: 1
camera_id: -1
cpu_threads: 1
device: CPU
enable_mkldnn: False
enable_mkldnn_bfloat16: False
image_dir: None
image_file: ./dataset/bamboo/testimg/1.jpg
model_dir: ./inference_model/yolov3_bamboo2
output_dir: output
random_pad: False
reid_batch_size: 50
reid_model_dir: None
run_benchmark: False
run_mode: paddle
save_images: False
save_mot_txt_per_img: False
save_mot_txts: False
scaled: False
threshold: 0.5
tracker_config: None
trt_calib_mode: False
trt_max_shape: 1280
trt_min_shape: 1
trt_opt_shape: 640
use_dark: True
use_gpu: False
video_file: None
window_size: 50
------------------------------------------
----------- Model Configuration -----------
Model Arch: YOLO
Transform Order:
--transform op: Resize
--transform op: NormalizeImage
--transform op: Permute
--------------------------------------------
[WARNNING] No object detected.
save result to: output\1.jpg
Test iter 0
------------------ Inference Time Info ----------------------
total_time(ms): 1339.2, img_num: 1
average latency time(ms): 1339.20, QPS: 0.746714
preprocess_time(ms): 120.50, inference_time(ms): 1218.70, postprocess_time(ms): 0.00
您好， 我在对模型进行量化训练后，使用命令：
能够正确进行预测推理。
然而进行动转静导出：
之后 使用命令
python deploy/python/infer.py --model_dir=output_inference/ppyolo_r50vd_qat_pact --image_dir=demo/testdata/1却无法对图片进行预测，也没有报错，输出的图像中没有预测的标签和边框。
使用命令：
python deploy/python/infer.py --model_dir=output_inference/ppyolo_r50vd_qat_pact --video_file=demo/1.mp4对视频进行测试时，报如下错误