search

PaddlePaddle / PaddleDetection

Object Detection toolkit based on PaddlePaddle. It supports object detection, instance segmentation, multiple object tracking and real-time multi-person keypoint detection.
Apache License 2.0
12.81k stars 2.89k forks source link

win10中使用deploy/cpp代码的tensorrt模式下，无法找到生成的engine文件，每次运行都会重新生成。 #5628

Closed Peter-weng closed 2 years ago

Peter-weng commented 2 years ago

PaddleDetection team appreciate any suggestion or problem you delivered~

Checklist:

  1. 查找历史相关issue寻求解答/I have searched related issues but cannot get the expected help.
  2. 翻阅FAQ /I have read the FAQ documentation but cannot get the expected help.
  3. 确认bug是否在新版本里还未修复/The bug has not been fixed in the latest version.

描述问题/Describe the bug

A clear and concise description of what the bug is. 每次使用相同的命令运行，都需要等待engine文件生成，上一次生成的engine没有被使用到

发现刚生成engine后马上推理几次是很快的，但是时间有时候很慢有时候很快，大概四五次之后又会卡住生成engine

复现/Reproduction

  1. 您使用的命令是？/What command or script did you run?
main.exe --model_dir=D:\\cpp\\yolov3_darknet53_270e_coco --image_file=1.jpg --device=GPU --run_mode=trt_fp16
  1. 您是否更改过代码或配置文件？您是否理解您所更改的内容？还请您提供所更改的部分代码。/Did you make any modifications on the code or config? Did you understand what you have modified? Please provide the codes that you modified. 修改了config.EnableTensorRtEngine(1 << 30, batch_size, this->min_subgraphsize, precision, true, this->trt_calibmode);
  2. 您使用的数据集是？/What dataset did you use? 默认yolov3_darknet53_270e_coco的模型
  3. 请提供您出现的报错信息及相关log。/Please provide the error messages or relevant log information. total images = 1, batch_size = 1, total steps = 1 class=0 confidence=0.9785 rect=[505 116 595 270] class=0 confidence=0.9756 rect=[417 89 496 285] class=0 confidence=0.9517 rect=[161 82 197 165] class=0 confidence=0.9219 rect=[580 114 613 200] class=0 confidence=0.8662 rect=[104 45 125 93] class=0 confidence=0.7949 rect=[348 45 364 99] class=0 confidence=0.6763 rect=[268 90 293 167] class=0 confidence=0.6709 rect=[366 59 381 113] class=0 confidence=0.5356 rect=[376 39 394 81] class=0 confidence=0.5303 rect=[26 118 57 151] class=33 confidence=0.9214 rect=[141 96 629 346] class=56 confidence=0.8179 rect=[73 121 102 152] 1.jpg The number of detected box: 12 Visualized output saved as output\1.jpg WARNING: Logging before InitGoogleLogging() is written to STDERR I0408 09:13:23.273309 13272 main_keypoint.cc:83] ----------------------- Config info ----------------------- I0408 09:13:23.274312 13272 main_keypoint.cc:84] runtime_device: GPU I0408 09:13:23.274312 13272 main_keypoint.cc:85] ir_optim: True I0408 09:13:23.274312 13272 main_keypoint.cc:87] enable_memory_optim: True I0408 09:13:23.274312 13272 main_keypoint.cc:91] enable_tensorrt: True I0408 09:13:23.274312 13272 main_keypoint.cc:94] precision: fp16 I0408 09:13:23.274312 13272 main_keypoint.cc:101] enable_mkldnn: False I0408 09:13:23.274312 13272 main_keypoint.cc:102] cpu_math_library_num_threads: 1 I0408 09:13:23.274312 13272 main_keypoint.cc:103] ----------------------- Data info ----------------------- I0408 09:13:23.275311 13272 main_keypoint.cc:104] batch_size: 1 I0408 09:13:23.275311 13272 main_keypoint.cc:105] input_shape: dynamic shape I0408 09:13:23.275311 13272 main_keypoint.cc:107] ----------------------- Model info ----------------------- I0408 09:13:23.275311 13272 main_keypoint.cc:109] model_name: D:\cpp\yolov3_darknet53_270e_coco I0408 09:13:23.275311 13272 main_keypoint.cc:110] ----------------------- Perf info ------------------------ I0408 09:13:23.276310 13272 main_keypoint.cc:111] Total number of predicted data: 1 and total time spent(ms): 209.994 I0408 09:13:23.276310 13272 main_keypoint.cc:115] preproce_time(ms): 8.0645, inference_time(ms): 201.907, postprocess_time(ms): 0.0225 上下两次指令都一样的 有时候测会很快就推理出来，像是加载了engine，但是时间不对，而且多试几次又不行了，又会去生成engine。 total images = 1, batch_size = 1, total steps = 1 class=0 confidence=0.9785 rect=[505 116 595 270] class=0 confidence=0.9756 rect=[417 89 496 285] class=0 confidence=0.9517 rect=[161 82 197 165] class=0 confidence=0.9219 rect=[580 114 613 200] class=0 confidence=0.8662 rect=[104 45 125 93] class=0 confidence=0.7949 rect=[348 45 364 99] class=0 confidence=0.6763 rect=[268 90 293 167] class=0 confidence=0.6709 rect=[366 59 381 113] class=0 confidence=0.5356 rect=[376 39 394 81] class=0 confidence=0.5303 rect=[26 118 57 151] class=33 confidence=0.9214 rect=[141 96 629 346] class=56 confidence=0.8179 rect=[73 121 102 152] 1.jpg The number of detected box: 12 Visualized output saved as output\1.jpg WARNING: Logging before InitGoogleLogging() is written to STDERR I0408 09:18:41.851187 6464 main_keypoint.cc:83] ----------------------- Config info ----------------------- I0408 09:18:41.851187 6464 main_keypoint.cc:84] runtime_device: GPU I0408 09:18:41.852198 6464 main_keypoint.cc:85] ir_optim: True I0408 09:18:41.852198 6464 main_keypoint.cc:87] enable_memory_optim: True I0408 09:18:41.852198 6464 main_keypoint.cc:91] enable_tensorrt: True I0408 09:18:41.852198 6464 main_keypoint.cc:94] precision: fp16 I0408 09:18:41.852198 6464 main_keypoint.cc:101] enable_mkldnn: False I0408 09:18:41.853195 6464 main_keypoint.cc:102] cpu_math_library_num_threads: 1 I0408 09:18:41.853195 6464 main_keypoint.cc:103] ----------------------- Data info ----------------------- I0408 09:18:41.853195 6464 main_keypoint.cc:104] batch_size: 1 I0408 09:18:41.853195 6464 main_keypoint.cc:105] input_shape: dynamic shape I0408 09:18:41.853195 6464 main_keypoint.cc:107] ----------------------- Model info ----------------------- I0408 09:18:41.853195 6464 main_keypoint.cc:109] model_name: D:\cpp\yolov3_darknet53_270e_coco I0408 09:18:41.854192 6464 main_keypoint.cc:110] ----------------------- Perf info ------------------------ I0408 09:18:41.854192 6464 main_keypoint.cc:111] Total number of predicted data: 1 and total time spent(ms): 970.951 I0408 09:18:41.854192 6464 main_keypoint.cc:115] preproce_time(ms): 7.4447, inference_time(ms): 963.49, postprocess_time(ms): 0.0162

total images = 1, batch_size = 1, total steps = 1 class=0 confidence=0.9780 rect=[505 116 595 270] class=0 confidence=0.9756 rect=[417 89 496 285] class=0 confidence=0.9517 rect=[161 82 197 165] class=0 confidence=0.9229 rect=[580 114 613 200] class=0 confidence=0.8657 rect=[104 45 125 93] class=0 confidence=0.7949 rect=[348 45 364 99] class=0 confidence=0.6748 rect=[268 90 293 167] class=0 confidence=0.6724 rect=[366 59 381 113] class=0 confidence=0.5352 rect=[376 39 394 81] class=0 confidence=0.5293 rect=[26 118 57 151] class=33 confidence=0.9219 rect=[140 96 630 346] class=56 confidence=0.8184 rect=[73 121 102 152] 1.jpg The number of detected box: 12 Visualized output saved as output\1.jpg WARNING: Logging before InitGoogleLogging() is written to STDERR I0408 09:47:24.132450 8024 main_keypoint.cc:83] ----------------------- Config info ----------------------- I0408 09:47:24.132450 8024 main_keypoint.cc:84] runtime_device: GPU I0408 09:47:24.132450 8024 main_keypoint.cc:85] ir_optim: True I0408 09:47:24.132450 8024 main_keypoint.cc:87] enable_memory_optim: True I0408 09:47:24.133487 8024 main_keypoint.cc:91] enable_tensorrt: True I0408 09:47:24.133487 8024 main_keypoint.cc:94] precision: fp16 I0408 09:47:24.134452 8024 main_keypoint.cc:101] enable_mkldnn: False I0408 09:47:24.134452 8024 main_keypoint.cc:102] cpu_math_library_num_threads: 1 I0408 09:47:24.135455 8024 main_keypoint.cc:103] ----------------------- Data info ----------------------- I0408 09:47:24.135455 8024 main_keypoint.cc:104] batch_size: 1 I0408 09:47:24.138485 8024 main_keypoint.cc:105] input_shape: dynamic shape I0408 09:47:24.139456 8024 main_keypoint.cc:107] ----------------------- Model info ----------------------- I0408 09:47:24.139456 8024 main_keypoint.cc:109] model_name: D:\cpp\yolov3_darknet53_270e_coco I0408 09:47:24.139456 8024 main_keypoint.cc:110] ----------------------- Perf info ------------------------ I0408 09:47:24.140453 8024 main_keypoint.cc:111] Total number of predicted data: 1 and total time spent(ms): 247.999 I0408 09:47:24.140453 8024 main_keypoint.cc:115] preproce_time(ms): 7.9415, inference_time(ms): 240.039, postprocess_time(ms): 0.0187

total images = 1, batch_size = 1, total steps = 1 class=0 confidence=0.9780 rect=[505 116 595 270] class=0 confidence=0.9756 rect=[417 89 496 285] class=0 confidence=0.9517 rect=[161 82 197 165] class=0 confidence=0.9229 rect=[580 114 613 200] class=0 confidence=0.8657 rect=[104 45 125 93] class=0 confidence=0.7949 rect=[348 45 364 99] class=0 confidence=0.6748 rect=[268 90 293 167] class=0 confidence=0.6724 rect=[366 59 381 113] class=0 confidence=0.5352 rect=[376 39 394 81] class=0 confidence=0.5293 rect=[26 118 57 151] class=33 confidence=0.9219 rect=[140 96 630 346] class=56 confidence=0.8184 rect=[73 121 102 152] 1.jpg The number of detected box: 12 Visualized output saved as output\1.jpg WARNING: Logging before InitGoogleLogging() is written to STDERR I0408 09:47:51.269739 13512 main_keypoint.cc:83] ----------------------- Config info ----------------------- I0408 09:47:51.269739 13512 main_keypoint.cc:84] runtime_device: GPU I0408 09:47:51.269739 13512 main_keypoint.cc:85] ir_optim: True I0408 09:47:51.269739 13512 main_keypoint.cc:87] enable_memory_optim: True I0408 09:47:51.269739 13512 main_keypoint.cc:91] enable_tensorrt: True I0408 09:47:51.270730 13512 main_keypoint.cc:94] precision: fp16 I0408 09:47:51.270730 13512 main_keypoint.cc:101] enable_mkldnn: False I0408 09:47:51.270730 13512 main_keypoint.cc:102] cpu_math_library_num_threads: 1 I0408 09:47:51.270730 13512 main_keypoint.cc:103] ----------------------- Data info ----------------------- I0408 09:47:51.271729 13512 main_keypoint.cc:104] batch_size: 1 I0408 09:47:51.271729 13512 main_keypoint.cc:105] input_shape: dynamic shape I0408 09:47:51.271729 13512 main_keypoint.cc:107] ----------------------- Model info ----------------------- I0408 09:47:51.271729 13512 main_keypoint.cc:109] model_name: D:\cpp\yolov3_darknet53_270e_coco I0408 09:47:51.271729 13512 main_keypoint.cc:110] ----------------------- Perf info ------------------------ I0408 09:47:51.271729 13512 main_keypoint.cc:111] Total number of predicted data: 1 and total time spent(ms): 33.0925 I0408 09:47:51.271729 13512 main_keypoint.cc:115] preproce_time(ms): 7.2268, inference_time(ms): 25.8489, postprocess_time(ms): 0.0168

total images = 1, batch_size = 1, total steps = 1 class=0 confidence=0.9780 rect=[505 116 595 270] class=0 confidence=0.9756 rect=[417 89 496 285] class=0 confidence=0.9517 rect=[161 82 197 165] class=0 confidence=0.9229 rect=[580 114 613 200] class=0 confidence=0.8657 rect=[104 45 125 93] class=0 confidence=0.7949 rect=[348 45 364 99] class=0 confidence=0.6748 rect=[268 90 293 167] class=0 confidence=0.6724 rect=[366 59 381 113] class=0 confidence=0.5352 rect=[376 39 394 81] class=0 confidence=0.5293 rect=[26 118 57 151] class=33 confidence=0.9219 rect=[140 96 630 346] class=56 confidence=0.8184 rect=[73 121 102 152] 1.jpg The number of detected box: 12 Visualized output saved as output\1.jpg WARNING: Logging before InitGoogleLogging() is written to STDERR I0408 09:48:12.210750 8736 main_keypoint.cc:83] ----------------------- Config info ----------------------- I0408 09:48:12.211752 8736 main_keypoint.cc:84] runtime_device: GPU I0408 09:48:12.211752 8736 main_keypoint.cc:85] ir_optim: True I0408 09:48:12.211752 8736 main_keypoint.cc:87] enable_memory_optim: True I0408 09:48:12.212751 8736 main_keypoint.cc:91] enable_tensorrt: True I0408 09:48:12.212751 8736 main_keypoint.cc:94] precision: fp16 I0408 09:48:12.212751 8736 main_keypoint.cc:101] enable_mkldnn: False I0408 09:48:12.212751 8736 main_keypoint.cc:102] cpu_math_library_num_threads: 1 I0408 09:48:12.213752 8736 main_keypoint.cc:103] ----------------------- Data info ----------------------- I0408 09:48:12.213752 8736 main_keypoint.cc:104] batch_size: 1 I0408 09:48:12.213752 8736 main_keypoint.cc:105] input_shape: dynamic shape I0408 09:48:12.213752 8736 main_keypoint.cc:107] ----------------------- Model info ----------------------- I0408 09:48:12.213752 8736 main_keypoint.cc:109] model_name: D:\cpp\yolov3_darknet53_270e_coco I0408 09:48:12.213752 8736 main_keypoint.cc:110] ----------------------- Perf info ------------------------ I0408 09:48:12.213752 8736 main_keypoint.cc:111] Total number of predicted data: 1 and total time spent(ms): 111.552 I0408 09:48:12.213752 8736 main_keypoint.cc:115] preproce_time(ms): 7.167, inference_time(ms): 104.37, postprocess_time(ms): 0.0144 捕获 上面是生成两次engine的情况，第一次三个，第二次好像只生成了两个新的，记得上次好像生成了三个新的。。

环境/Environment

  1. 请提供您使用的Paddle和PaddleDetection的版本号/Please provide the version of Paddle and PaddleDetection you use： paddledet 2.4.0 paddlepaddle-gpu 2.2.2.post112 部署在win10上并没有环境，环境是在Linux上配的，应该没影响吧

  2. 如您在使用PaddleDetection的同时还在使用其他产品，如PaddleServing、PaddleInference等，请您提供其版本号/ Please provide the version of any other related tools/products used, such as the version of PaddleServing and etc：

  3. 请提供您使用的操作系统信息，如Linux/Windows/MacOS /Please provide the OS information, e.g., Linux： Windows

  4. 请问您使用的Python版本是？/ Please provide the version of Python you used. 导出用的是3.9.11

  5. 请问您使用的CUDA/cuDNN的版本号是？/ Please provide the version of CUDA/cuDNN you used. win10中11.3和8.3.2

如果您的issue是关于安装或环境，您可以先查询安装文档尝试解决~

If your issue looks like an installation issue / environment issue, please first try to solve it yourself with the instructions in https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.1/docs/tutorials/INSTALL.md

TTgogogo commented 2 years ago

似乎很多人发现这个问题了

jerrywgz commented 2 years ago

这是windows下特有的问题，我们现在在高优解决了，有结论第一时间同步出来

ZhiluDing commented 2 years ago

不仅windows上出现了，再jetson nano上面也出现这个问题了，序列化的模型总是重新生成