Does gen_wts_yoloV5.py support the improved model?

[chengzihencai]

I try to run python3 gen_wts_yoloV5.py -w yolov5s.pt (the yolov5s.pt was trained by yolov5 with a modified backbone and neck) ，the output is 'Model not supported'. Although I got yolov5.cfg and yolov5s.wts，the yolov5s.wts is empty,the yolov5s.wts is almost empty [net] width=640 height=640 channels=3 letter_box=1

Is there any way to generate cfg wts engine with a modified backbone and neck-yolov5?

[marcoslucianops]

I need to know the layers to implement it on gen_wts_yoloV5.py. I created the code only for YOLOv5 >= 2.0 layers.

[chengzihencai]

It is my models:`

             from  n    params  module                                  arguments                     

0 -1 1 928 models.common.conv_bn_relu_maxpool [3, 32]
1 -1 1 8812 models.common.Shuffle_Block [32, 120, 2]
2 -1 3 24300 models.common.Shuffle_Block [120, 120, 1]
3 -1 1 44588 models.common.Shuffle_Block [120, 232, 2]
4 -1 7 200564 models.common.Shuffle_Block [232, 232, 1]
5 -1 1 167968 models.common.Shuffle_Block [232, 464, 2]
6 -1 1 111128 models.common.Shuffle_Block [464, 464, 1]
7 -1 1 44736 models.common.Conv [464, 96, 1, 1]
8 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
9 [-1, 4] 1 0 models.common.Concat [1]
10 -1 1 35288 models.common.DWConvblock [328, 96, 3, 1]
11 -1 1 9408 models.common.Conv [96, 96, 1, 1]
12 -1 1 0 torch.nn.modules.upsampling.Upsample [None, 2, 'nearest']
13 [-1, 2] 1 0 models.common.Concat [1]
14 -1 1 23304 models.common.DWConvblock [216, 96, 3, 1]
15 -1 1 10464 models.common.DWConvblock [96, 96, 3, 2]
16 [-1, 11] 1 0 models.common.Concat [1]
17 -1 1 20736 models.common.DWConvblock [192, 96, 3, 1]
18 -1 1 10464 models.common.DWConvblock [96, 96, 3, 2]
19 [-1, 7] 1 0 models.common.Concat [1]
20 -1 1 20736 models.common.DWConvblock [192, 96, 3, 1]
21 [14, 17, 20] 1 14841 models.yolo.Detect [12, [[10, 13, 16, 30, 33, 23], [30, 61, 62, 45, 59, 119], [116, 90, 156, 198, 373, 326]], [96, 96, 96]]

What should I do to makegen_wts_yoloV5.py` support the operation of this model？

[marcoslucianops]

I will need the model and the edited layers (common file) to do it. If you can share it, you can send me via email.

[chengzihencai]

I send them to your email,many thanks for your help!

[chengzihencai]

Hi,about this issue，I had sent my model and common file to your email,can I get your help?My email adress is caiOrange@outlook.com

[marcoslucianops]

I received the files, I'll work on it next week. I'm full of work this week. I removed the duplicated comment.

[chengzihencai]

Thank you most sincerely.

[marcoslucianops]

I added support for the model you sent me in the email. Please test with the new gen_wts_yoloV5.py file.

[chengzihencai]

First, thanks abouts it!
I tested the project with the new gen_wts_yoloV5 file in these days,and no errors were encountered.But in the end I tested project "DeepStream-Yoolo-master" with an H264 mp4 file, there were no targets (including the right target and the wrong target) in the screen, what was the reason for this? Here is my operation process： ` step1. Rename my .pt file from best.pt to yolvo5s.pt step2. Copy gen_wts_yoloV5.py to file floder yolov5 step3. Open gen_wts_yolov5.py and add these to the top of the file:

import matplotlib matplotlib.use('Agg')

Otherwise, an error will be reported: Unable to init server: Could not connect: Connection refused Unable to init server: Could not connect: Connection refused (gen_wts_yoloV5.py:12368): Gdk-CRITICAL **: 09:09:12.983: gdk_cursor_new_for_display: assertion 'GDK_IS_DISPLAY (display)' failed

step4. Generate the cfg and wts files: cd yolov5 and run:python3 gen_wts_yoloV5.py -w yolov5s.pt step5. Copy the generated cfg and wts files to the DeepStream-Yolo folder. step6. Open the DeepStream-Yolo and run CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo step7. Edit labels.txt step8. Edit num-detected-classes in config_infer_primary_yoloV5.txt step9. Edit the deepstream_app_config file : config-file=config_infer_primary_yoloV5.txt step10. export DISPLAY=:0 or report error: Could not open X Display step11. deepstream-app -c deepstream_app_config.txt`

And log: `orange@orange-desktop:~/Desktop/DeepStream-Yolo-master$ export DISPLAY=:0 orange@orange-desktop:~/Desktop/DeepStream-Yolo-master$ deepstream-app -c deepstream_app_config.txt

Using winsys: x11 ERROR: Deserialize engine failed because file path: /home/lcfc/Desktop/DeepStream-Yolo-master/model_b1_gpu0_fp32.engine open error 0:00:01.752069024 19996 0x270cc8f0 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() [UID = 1]: deserialize engine from file :/home/lcfc/Desktop/DeepStream-Yolo-master/model_b1_gpu0_fp32.engine failed 0:00:01.752340736 19996 0x270cc8f0 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() [UID = 1]: deserialize backend context from engine from file :/home/lcfc/Desktop/DeepStream-Yolo-master/model_b1_gpu0_fp32.engine failed, try rebuild 0:00:01.752410144 19996 0x270cc8f0 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() [UID = 1]: Trying to create engine from model files

Loading pre-trained weights Loading weights of yolov5s complete Total weights read: 4988377 Building YOLO network

    Layer                         Input Shape         Output Shape        WeightPtr

(0) conv_linear [3, 640, 640] [32, 320, 320] 864 (1) conv_hardswish [32, 320, 320] [32, 160, 160] 1280 (2) conv_hardswish [32, 160, 160] [64, 160, 160] 3584 (3) conv_hardswish [64, 160, 160] [64, 160, 160] 4416 (4) conv_hardswish [64, 160, 160] [64, 160, 160] 8768 (5) conv_hardswish [64, 160, 160] [64, 80, 80] 9600 (6) conv_hardswish [64, 80, 80] [128, 80, 80] 18304 (7) conv_hardswish [128, 80, 80] [128, 80, 80] 19968 (8) conv_hardswish [128, 80, 80] [128, 80, 80] 36864 (9) conv_hardswish [128, 80, 80] [128, 80, 80] 38528 (10) conv_hardswish [128, 80, 80] [128, 80, 80] 55424 (11) conv_hardswish [128, 80, 80] [128, 80, 80] 57088 (12) conv_hardswish [128, 80, 80] [128, 80, 80] 73984 (13) conv_hardswish [128, 80, 80] [128, 40, 40] 75648 (14) conv_hardswish [128, 40, 40] [256, 40, 40] 109440 (15) conv_hardswish [256, 40, 40] [256, 40, 40] 116864 (16) conv_hardswish [256, 40, 40] [256, 40, 40] 183424 (17) conv_hardswish [256, 40, 40] [256, 40, 40] 190848 (18) conv_hardswish [256, 40, 40] [256, 40, 40] 257408 (19) conv_hardswish [256, 40, 40] [256, 40, 40] 264832 (20) conv_hardswish [256, 40, 40] [256, 40, 40] 331392 (21) conv_hardswish [256, 40, 40] [256, 40, 40] 338816 (22) conv_hardswish [256, 40, 40] [256, 40, 40] 405376 (23) conv_hardswish [256, 40, 40] [256, 40, 40] 412800 (24) conv_hardswish [256, 40, 40] [256, 40, 40] 479360 (25) conv_hardswish [256, 40, 40] [256, 20, 20] 486784 (26) avgpool [256, 20, 20] [256, 1, 1] - (27) conv_linear [256, 1, 1] [64, 1, 1] 503232 (28) conv_linear [64, 1, 1] [256, 1, 1] 519872 (29) shortcut_mul_linear: 25 [256, 1, 1] [256, 20, 20] - (30) conv_hardswish [256, 20, 20] [512, 20, 20] 652992 (31) conv_hardswish [512, 20, 20] [512, 20, 20] 667840 (32) avgpool [512, 20, 20] [512, 1, 1] - (33) conv_linear [512, 1, 1] [128, 1, 1] 733504 (34) conv_linear [128, 1, 1] [512, 1, 1] 799552 (35) shortcut_mul_linear: 31 [512, 1, 1] [512, 20, 20] - (36) conv_hardswish [512, 20, 20] [512, 20, 20] 1063744 (37) conv_hardswish [512, 20, 20] [512, 20, 20] 1078592 (38) avgpool [512, 20, 20] [512, 1, 1] - (39) conv_linear [512, 1, 1] [128, 1, 1] 1144256 (40) conv_linear [128, 1, 1] [512, 1, 1] 1210304 (41) shortcut_mul_linear: 37 [512, 1, 1] [512, 20, 20] - (42) conv_hardswish [512, 20, 20] [512, 20, 20] 1474496 (43) conv_hardswish [512, 20, 20] [512, 20, 20] 1489344 (44) avgpool [512, 20, 20] [512, 1, 1] - (45) conv_linear [512, 1, 1] [128, 1, 1] 1555008 (46) conv_linear [128, 1, 1] [512, 1, 1] 1621056 (47) shortcut_mul_linear: 43 [512, 1, 1] [512, 20, 20] - (48) conv_hardswish [512, 20, 20] [512, 20, 20] 1885248 (49) conv_linear [512, 20, 20] [512, 20, 20] 2147392 (50) conv_silu [512, 20, 20] [256, 20, 20] 2279488 (51) upsample [256, 20, 20] [256, 40, 40] - (52) route: 51, 24 - [512, 40, 40] - (53) conv_silu [512, 40, 40] [128, 40, 40] 2345536 (54) route: 52 - [512, 40, 40] - (55) conv_silu [512, 40, 40] [128, 40, 40] 2411584 (56) conv_silu [128, 40, 40] [128, 40, 40] 2428480 (57) conv_silu [128, 40, 40] [128, 40, 40] 2576448 (58) route: 57, 53 - [256, 40, 40] - (59) conv_silu [256, 40, 40] [256, 40, 40] 2643008 (60) conv_silu [256, 40, 40] [256, 40, 40] 2709568 (61) upsample [256, 40, 40] [256, 80, 80] - (62) route: 61, 12 - [384, 80, 80] - (63) conv_silu [384, 80, 80] [128, 80, 80] 2759232 (64) route: 62 - [384, 80, 80] - (65) conv_silu [384, 80, 80] [128, 80, 80] 2808896 (66) conv_silu [128, 80, 80] [128, 80, 80] 2825792 (67) conv_silu [128, 80, 80] [128, 80, 80] 2973760 (68) route: 67, 63 - [256, 80, 80] - (69) conv_silu [256, 80, 80] [256, 80, 80] 3040320 (70) conv_silu [256, 80, 80] [256, 40, 40] 3631168 (71) route: 70, 60 - [512, 40, 40] - (72) conv_silu [512, 40, 40] [128, 40, 40] 3697216 (73) route: 71 - [512, 40, 40] - (74) conv_silu [512, 40, 40] [128, 40, 40] 3763264 (75) conv_silu [128, 40, 40] [128, 40, 40] 3780160 (76) conv_silu [128, 40, 40] [128, 40, 40] 3928128 (77) route: 76, 72 - [256, 40, 40] - (78) conv_silu [256, 40, 40] [256, 40, 40] 3994688 (79) conv_silu [256, 40, 40] [256, 20, 20] 4585536 (80) route: 79, 50 - [512, 20, 20] - (81) conv_silu [512, 20, 20] [128, 20, 20] 4651584 (82) route: 80 - [512, 20, 20] - (83) conv_silu [512, 20, 20] [128, 20, 20] 4717632 (84) conv_silu [128, 20, 20] [128, 20, 20] 4734528 (85) conv_silu [128, 20, 20] [128, 20, 20] 4882496 (86) route: 85, 81 - [256, 20, 20] - (87) conv_silu [256, 20, 20] [256, 20, 20] 4949056 (88) route: 69 - [256, 80, 80] - (89) conv_logistic [256, 80, 80] [51, 80, 80] 4962163 (90) yolo [51, 80, 80] - - (91) route: 78 - [256, 40, 40] - (92) conv_logistic [256, 40, 40] [51, 40, 40] 4975270 (93) yolo [51, 40, 40] - - (94) route: 87 - [256, 20, 20] - (95) conv_logistic [256, 20, 20] [51, 20, 20] 4988377 (96) yolo [51, 20, 20] - -

Output YOLO blob names: yolo_91 yolo_94 yolo_97

Total number of YOLO layers: 259

Building YOLO network complete Building the TensorRT Engine

NOTE: letter_box is set in cfg file, make sure to set maintain-aspect-ratio=1 in config_infer file to get better accuracy

WARNING: [TRT]: Detected invalid timing cache, setup a local cache instead Building complete

0:01:58.450481056 19996 0x270cc8f0 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() [UID = 1]: serialize cuda engine to file: /home/lcfc/Desktop/DeepStream-Yolo-master/model_b1_gpu0_fp32.engine successfully INFO: [Implicit Engine Info]: layers num: 5 0 INPUT kFLOAT data 3x640x640
1 OUTPUT kFLOAT num_detections 1
2 OUTPUT kFLOAT detection_boxes 25200x4
3 OUTPUT kFLOAT detection_scores 25200
4 OUTPUT kFLOAT detection_classes 25200

0:01:58.692168416 19996 0x270cc8f0 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus: [UID 1]: Load new model:/home/lcfc/Desktop/DeepStream-Yolo-master/config_infer_primary_yoloV5.txt sucessfully

Runtime commands: h: Print this help q: Quit

p: Pause
r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source. To go back to the tiled display, right-click anywhere on the window.

PERF: FPS 0 (Avg)
PERF: 0.00 (0.00)
** INFO: : Pipeline ready

Opening in BLOCKING MODE NvMMLiteOpen : Block : BlockType = 261 NVMEDIA: Reading vendor.tegra.display-size : status: 6 NvMMLiteBlockCreate : Block : BlockType = 261 ** INFO: : Pipeline running

PERF: 19.48 (19.29)
PERF: 19.51 (19.47)
PERF: 19.65 (19.52)
INFO: : Received EOS. Exiting ...

Quitting App run successful` There's no error was reported,but I can't find target (right or wrong).I wonder how to solve this problem.

[marcoslucianops]

Can you send me one trained model to check?

[chengzihencai]

Sure, I've sent it to your email.

[marcoslucianops]

With the ONNX export, it should support the YOLOv5 model with custom layers.