search

marcoslucianops / DeepStream-Yolo

NVIDIA DeepStream SDK 7.0 / 6.4 / 6.3 / 6.2 / 6.1.1 / 6.1 / 6.0.1 / 6.0 / 5.1 implementation for YOLO models
MIT License
1.44k stars 354 forks source link

Inference parsing error #178

Closed madisi98 closed 2 years ago

madisi98 commented 2 years ago

Hello, I'm trying to use this inside a python app with the DeepStream python bindings. The script I'm running is this one:

https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/blob/master/apps/deepstream-rtsp-in-rtsp-out/deepstream_test1_rtsp_in_rtsp_out.py

I replaced the configuration pgie file for the one provided in this repo and I'm finding an error when 'parsing the output'. The output log is this:

`root@838991a79d83:/workspace/packages/deepstream_python_apps/apps/deepstream-rtsp-in-rtsp-out# python3 deepstream_test1_rtsp_in_rtsp_out.py -i rtsp://192.168.1.40:554/media/sniper.mkv Creating Pipeline

Creating streamux

Creating source_bin 0

Creating source bin source-bin-00 Creating Pgie

Creating tiler

Creating nvvidconv

Creating nvosd

Creating H264 Encoder Creating H264 rtppay Adding elements to Pipeline

DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test

Starting pipeline

WARNING: ../nvdsinfer/nvdsinfer_model_builder.cpp:1482 Deserialize engine failed because file path: /workspace/packages/yolor/yolor/model.prueba-batch1-32.engine open error

Loading pre-trained weights Loading weights of /workspace/packages/yolor/yolor/cfg/yolor_p6 complete Total weights read: 37329400 Building YOLO network

  layer                        input               output         weightPtr

(0) reorgV5 3 x1280 x1280 12 x 640 x 640 0
(1) conv_silu 12 x 640 x 640 64 x 640 x 640 7168
(2) conv_silu 64 x 640 x 640 128 x 320 x 320 81408
(3) conv_silu 128 x 320 x 320 64 x 320 x 320 89856
(4) route - 128 x 320 x 320 89856
(5) conv_silu 128 x 320 x 320 64 x 320 x 320 98304
(6) conv_silu 64 x 320 x 320 64 x 320 x 320 102656 (7) conv_silu 64 x 320 x 320 64 x 320 x 320 139776 (8) shortcut_linear: 5 - 64 x 320 x 320 -
(9) conv_silu 64 x 320 x 320 64 x 320 x 320 144128 (10) conv_silu 64 x 320 x 320 64 x 320 x 320 181248 (11) shortcut_linear: 8 - 64 x 320 x 320 -
(12) conv_silu 64 x 320 x 320 64 x 320 x 320 185600 (13) conv_silu 64 x 320 x 320 64 x 320 x 320 222720 (14) shortcut_linear: 11 - 64 x 320 x 320 -
(15) route - 128 x 320 x 320 222720 (16) conv_silu 128 x 320 x 320 128 x 320 x 320 239616 (17) conv_silu 128 x 320 x 320 256 x 160 x 160 535552 (18) conv_silu 256 x 160 x 160 128 x 160 x 160 568832 (19) route - 256 x 160 x 160 568832 (20) conv_silu 256 x 160 x 160 128 x 160 x 160 602112 (21) conv_silu 128 x 160 x 160 128 x 160 x 160 619008 (22) conv_silu 128 x 160 x 160 128 x 160 x 160 766976 (23) shortcut_linear: 20 - 128 x 160 x 160 -
(24) conv_silu 128 x 160 x 160 128 x 160 x 160 783872 (25) conv_silu 128 x 160 x 160 128 x 160 x 160 931840 (26) shortcut_linear: 23 - 128 x 160 x 160 -
(27) conv_silu 128 x 160 x 160 128 x 160 x 160 948736 (28) conv_silu 128 x 160 x 160 128 x 160 x 160 1096704 (29) shortcut_linear: 26 - 128 x 160 x 160 -
(30) conv_silu 128 x 160 x 160 128 x 160 x 160 1113600 (31) conv_silu 128 x 160 x 160 128 x 160 x 160 1261568 (32) shortcut_linear: 29 - 128 x 160 x 160 -
(33) conv_silu 128 x 160 x 160 128 x 160 x 160 1278464 (34) conv_silu 128 x 160 x 160 128 x 160 x 160 1426432 (35) shortcut_linear: 32 - 128 x 160 x 160 -
(36) conv_silu 128 x 160 x 160 128 x 160 x 160 1443328 (37) conv_silu 128 x 160 x 160 128 x 160 x 160 1591296 (38) shortcut_linear: 35 - 128 x 160 x 160 -
(39) conv_silu 128 x 160 x 160 128 x 160 x 160 1608192 (40) conv_silu 128 x 160 x 160 128 x 160 x 160 1756160 (41) shortcut_linear: 38 - 128 x 160 x 160 -
(42) route - 256 x 160 x 160 1756160 (43) conv_silu 256 x 160 x 160 256 x 160 x 160 1822720 (44) conv_silu 256 x 160 x 160 384 x 80 x 80 2708992 (45) conv_silu 384 x 80 x 80 192 x 80 x 80 2783488 (46) route - 384 x 80 x 80 2783488 (47) conv_silu 384 x 80 x 80 192 x 80 x 80 2857984 (48) conv_silu 192 x 80 x 80 192 x 80 x 80 2895616 (49) conv_silu 192 x 80 x 80 192 x 80 x 80 3228160 (50) shortcut_linear: 47 - 192 x 80 x 80 -
(51) conv_silu 192 x 80 x 80 192 x 80 x 80 3265792 (52) conv_silu 192 x 80 x 80 192 x 80 x 80 3598336 (53) shortcut_linear: 50 - 192 x 80 x 80 -
(54) conv_silu 192 x 80 x 80 192 x 80 x 80 3635968 (55) conv_silu 192 x 80 x 80 192 x 80 x 80 3968512 (56) shortcut_linear: 53 - 192 x 80 x 80 -
(57) conv_silu 192 x 80 x 80 192 x 80 x 80 4006144 (58) conv_silu 192 x 80 x 80 192 x 80 x 80 4338688 (59) shortcut_linear: 56 - 192 x 80 x 80 -
(60) conv_silu 192 x 80 x 80 192 x 80 x 80 4376320 (61) conv_silu 192 x 80 x 80 192 x 80 x 80 4708864 (62) shortcut_linear: 59 - 192 x 80 x 80 -
(63) conv_silu 192 x 80 x 80 192 x 80 x 80 4746496 (64) conv_silu 192 x 80 x 80 192 x 80 x 80 5079040 (65) shortcut_linear: 62 - 192 x 80 x 80 -
(66) conv_silu 192 x 80 x 80 192 x 80 x 80 5116672 (67) conv_silu 192 x 80 x 80 192 x 80 x 80 5449216 (68) shortcut_linear: 65 - 192 x 80 x 80 -
(69) route - 384 x 80 x 80 5449216 (70) conv_silu 384 x 80 x 80 384 x 80 x 80 5598208 (71) conv_silu 384 x 80 x 80 512 x 40 x 40 7369728 (72) conv_silu 512 x 40 x 40 256 x 40 x 40 7501824 (73) route - 512 x 40 x 40 7501824 (74) conv_silu 512 x 40 x 40 256 x 40 x 40 7633920 (75) conv_silu 256 x 40 x 40 256 x 40 x 40 7700480 (76) conv_silu 256 x 40 x 40 256 x 40 x 40 8291328 (77) shortcut_linear: 74 - 256 x 40 x 40 -
(78) conv_silu 256 x 40 x 40 256 x 40 x 40 8357888 (79) conv_silu 256 x 40 x 40 256 x 40 x 40 8948736 (80) shortcut_linear: 77 - 256 x 40 x 40 -
(81) conv_silu 256 x 40 x 40 256 x 40 x 40 9015296 (82) conv_silu 256 x 40 x 40 256 x 40 x 40 9606144 (83) shortcut_linear: 80 - 256 x 40 x 40 -
(84) route - 512 x 40 x 40 9606144 (85) conv_silu 512 x 40 x 40 512 x 40 x 40 9870336 (86) conv_silu 512 x 40 x 40 640 x 20 x 20 12822016 (87) conv_silu 640 x 20 x 20 320 x 20 x 20 13028096 (88) route - 640 x 20 x 20 13028096 (89) conv_silu 640 x 20 x 20 320 x 20 x 20 13234176 (90) conv_silu 320 x 20 x 20 320 x 20 x 20 13337856 (91) conv_silu 320 x 20 x 20 320 x 20 x 20 14260736 (92) shortcut_linear: 89 - 320 x 20 x 20 -
(93) conv_silu 320 x 20 x 20 320 x 20 x 20 14364416 (94) conv_silu 320 x 20 x 20 320 x 20 x 20 15287296 (95) shortcut_linear: 92 - 320 x 20 x 20 -
(96) conv_silu 320 x 20 x 20 320 x 20 x 20 15390976 (97) conv_silu 320 x 20 x 20 320 x 20 x 20 16313856 (98) shortcut_linear: 95 - 320 x 20 x 20 -
(99) route - 640 x 20 x 20 16313856 (100) conv_silu 640 x 20 x 20 640 x 20 x 20 16726016 (101) conv_silu 640 x 20 x 20 320 x 20 x 20 16932096 (102) route - 640 x 20 x 20 16932096 (103) conv_silu 640 x 20 x 20 320 x 20 x 20 17138176 (104) conv_silu 320 x 20 x 20 320 x 20 x 20 18061056 (105) conv_silu 320 x 20 x 20 320 x 20 x 20 18164736 (106) maxpool 320 x 20 x 20 320 x 20 x 20 18164736 (107) route - 320 x 20 x 20 18164736 (108) maxpool 320 x 20 x 20 320 x 20 x 20 18164736 (109) route - 320 x 20 x 20 18164736 (110) maxpool 320 x 20 x 20 320 x 20 x 20 18164736 (111) route - 1280 x 20 x 20 18164736 (112) conv_silu 1280 x 20 x 20 320 x 20 x 20 18575616 (113) conv_silu 320 x 20 x 20 320 x 20 x 20 19498496 (114) route - 640 x 20 x 20 19498496 (115) conv_silu 640 x 20 x 20 320 x 20 x 20 19704576 (116) conv_silu 320 x 20 x 20 256 x 20 x 20 19787520 (117) upsample 256 x 20 x 20 256 x 40 x 40 -
(118) route - 512 x 40 x 40 19787520 (119) conv_silu 512 x 40 x 40 256 x 40 x 40 19919616 (120) route - 512 x 40 x 40 19919616 (121) conv_silu 512 x 40 x 40 256 x 40 x 40 20051712 (122) conv_silu 256 x 40 x 40 256 x 40 x 40 20118272 (123) route - 256 x 40 x 40 20118272 (124) conv_silu 256 x 40 x 40 256 x 40 x 40 20184832 (125) conv_silu 256 x 40 x 40 256 x 40 x 40 20775680 (126) conv_silu 256 x 40 x 40 256 x 40 x 40 20842240 (127) conv_silu 256 x 40 x 40 256 x 40 x 40 21433088 (128) conv_silu 256 x 40 x 40 256 x 40 x 40 21499648 (129) conv_silu 256 x 40 x 40 256 x 40 x 40 22090496 (130) route - 512 x 40 x 40 22090496 (131) conv_silu 512 x 40 x 40 256 x 40 x 40 22222592 (132) conv_silu 256 x 40 x 40 192 x 40 x 40 22272512 (133) upsample 192 x 40 x 40 192 x 80 x 80 -
(134) route - 384 x 80 x 80 22272512 (135) conv_silu 384 x 80 x 80 192 x 80 x 80 22347008 (136) route - 384 x 80 x 80 22347008 (137) conv_silu 384 x 80 x 80 192 x 80 x 80 22421504 (138) conv_silu 192 x 80 x 80 192 x 80 x 80 22459136 (139) route - 192 x 80 x 80 22459136 (140) conv_silu 192 x 80 x 80 192 x 80 x 80 22496768 (141) conv_silu 192 x 80 x 80 192 x 80 x 80 22829312 (142) conv_silu 192 x 80 x 80 192 x 80 x 80 22866944 (143) conv_silu 192 x 80 x 80 192 x 80 x 80 23199488 (144) conv_silu 192 x 80 x 80 192 x 80 x 80 23237120 (145) conv_silu 192 x 80 x 80 192 x 80 x 80 23569664 (146) route - 384 x 80 x 80 23569664 (147) conv_silu 384 x 80 x 80 192 x 80 x 80 23644160 (148) conv_silu 192 x 80 x 80 128 x 80 x 80 23669248 (149) upsample 128 x 80 x 80 128 x 160 x 160 -
(150) route - 256 x 160 x 160 23669248 (151) conv_silu 256 x 160 x 160 128 x 160 x 160 23702528 (152) route - 256 x 160 x 160 23702528 (153) conv_silu 256 x 160 x 160 128 x 160 x 160 23735808 (154) conv_silu 128 x 160 x 160 128 x 160 x 160 23752704 (155) route - 128 x 160 x 160 23752704 (156) conv_silu 128 x 160 x 160 128 x 160 x 160 23769600 (157) conv_silu 128 x 160 x 160 128 x 160 x 160 23917568 (158) conv_silu 128 x 160 x 160 128 x 160 x 160 23934464 (159) conv_silu 128 x 160 x 160 128 x 160 x 160 24082432 (160) conv_silu 128 x 160 x 160 128 x 160 x 160 24099328 (161) conv_silu 128 x 160 x 160 128 x 160 x 160 24247296 (162) route - 256 x 160 x 160 24247296 (163) conv_silu 256 x 160 x 160 128 x 160 x 160 24280576 (164) conv_silu 128 x 160 x 160 192 x 80 x 80 24502528 (165) route - 384 x 80 x 80 24502528 (166) conv_silu 384 x 80 x 80 192 x 80 x 80 24577024 (167) conv_silu 192 x 80 x 80 192 x 80 x 80 24614656 (168) route - 192 x 80 x 80 24614656 (169) conv_silu 192 x 80 x 80 192 x 80 x 80 24652288 (170) conv_silu 192 x 80 x 80 192 x 80 x 80 24984832 (171) conv_silu 192 x 80 x 80 192 x 80 x 80 25022464 (172) conv_silu 192 x 80 x 80 192 x 80 x 80 25355008 (173) conv_silu 192 x 80 x 80 192 x 80 x 80 25392640 (174) conv_silu 192 x 80 x 80 192 x 80 x 80 25725184 (175) route - 384 x 80 x 80 25725184 (176) conv_silu 384 x 80 x 80 192 x 80 x 80 25799680 (177) conv_silu 192 x 80 x 80 256 x 40 x 40 26243072 (178) route - 512 x 40 x 40 26243072 (179) conv_silu 512 x 40 x 40 256 x 40 x 40 26375168 (180) conv_silu 256 x 40 x 40 256 x 40 x 40 26441728 (181) route - 256 x 40 x 40 26441728 (182) conv_silu 256 x 40 x 40 256 x 40 x 40 26508288 (183) conv_silu 256 x 40 x 40 256 x 40 x 40 27099136 (184) conv_silu 256 x 40 x 40 256 x 40 x 40 27165696 (185) conv_silu 256 x 40 x 40 256 x 40 x 40 27756544 (186) conv_silu 256 x 40 x 40 256 x 40 x 40 27823104 (187) conv_silu 256 x 40 x 40 256 x 40 x 40 28413952 (188) route - 512 x 40 x 40 28413952 (189) conv_silu 512 x 40 x 40 256 x 40 x 40 28546048 (190) conv_silu 256 x 40 x 40 320 x 20 x 20 29284608 (191) route - 640 x 20 x 20 29284608 (192) conv_silu 640 x 20 x 20 320 x 20 x 20 29490688 (193) conv_silu 320 x 20 x 20 320 x 20 x 20 29594368 (194) route - 320 x 20 x 20 29594368 (195) conv_silu 320 x 20 x 20 320 x 20 x 20 29698048 (196) conv_silu 320 x 20 x 20 320 x 20 x 20 30620928 (197) conv_silu 320 x 20 x 20 320 x 20 x 20 30724608 (198) conv_silu 320 x 20 x 20 320 x 20 x 20 31647488 (199) conv_silu 320 x 20 x 20 320 x 20 x 20 31751168 (200) conv_silu 320 x 20 x 20 320 x 20 x 20 32674048 (201) route - 640 x 20 x 20 32674048 (202) conv_silu 640 x 20 x 20 320 x 20 x 20 32880128 (203) implicit_add - 256 x 1 x 1 32880384 (204) implicit_add - 384 x 1 x 1 32880768 (205) implicit_add - 512 x 1 x 1 32881280 (206) implicit_add - 640 x 1 x 1 32881920 (207) implicit_mul - 255 x 1 x 1 32882175 (208) implicit_mul - 255 x 1 x 1 32882430 (209) implicit_mul - 255 x 1 x 1 32882685 (210) implicit_mul - 255 x 1 x 1 32882940 (211) route - 128 x 160 x 160 32882940 (212) conv_silu 128 x 160 x 160 256 x 160 x 160 33178876 (213) shift_channels: 203 - 256 x 160 x 160 -
(214) conv_linear 256 x 160 x 160 255 x 160 x 160 33244411 (215) control_channels: 207 - 255 x 160 x 160 -
(216) yolo 255 x 160 x 160 255 x 160 x 160 33244411 (217) route - 192 x 80 x 80 33244411 (218) conv_silu 192 x 80 x 80 384 x 80 x 80 33909499 (219) shift_channels: 204 - 384 x 80 x 80 -
(220) conv_linear 384 x 80 x 80 255 x 80 x 80 34007674 (221) control_channels: 208 - 255 x 80 x 80 -
(222) yolo 255 x 80 x 80 255 x 80 x 80 34007674 (223) route - 256 x 40 x 40 34007674 (224) conv_silu 256 x 40 x 40 512 x 40 x 40 35189370 (225) shift_channels: 205 - 512 x 40 x 40 -
(226) conv_linear 512 x 40 x 40 255 x 40 x 40 35320185 (227) control_channels: 209 - 255 x 40 x 40 -
(228) yolo 255 x 40 x 40 255 x 40 x 40 35320185 (229) route - 320 x 20 x 20 35320185 (230) conv_silu 320 x 20 x 20 640 x 20 x 20 37165945 (231) shift_channels: 206 - 640 x 20 x 20 -
(232) conv_linear 640 x 20 x 20 255 x 20 x 20 37329400 (233) control_channels: 210 - 255 x 20 x 20 -
(234) yolo 255 x 20 x 20 255 x 20 x 20 37329400 Output YOLO blob names: yolo_217 yolo_223 yolo_229 yolo_235 Total number of YOLO layers: 662 Building YOLO network complete Building the TensorRT Engine

Building complete

INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 5 0 INPUT kFLOAT data 3x1280x1280
1 OUTPUT kFLOAT yolo_217 255x160x160
2 OUTPUT kFLOAT yolo_223 255x80x80
3 OUTPUT kFLOAT yolo_229 255x40x40
4 OUTPUT kFLOAT yolo_235 255x20x20

ERROR: nvdsinfer_context_impl.cpp:589 parse label file:/model failed, nvinfer error:NVDSINFER_CONFIG_FAILED ERROR: nvdsinfer_context_impl.cpp:715 init post processing resource failed, nvinfer error:NVDSINFER_CONFIG_FAILED ERROR: nvdsinfer_context_impl.cpp:1065 Infer Context failed to initialize post-processing resource, nvinfer error:NVDSINFER_CONFIG_FAILED ERROR: nvdsinfer_context_impl.cpp:1271 Infer Context prepare postprocessing resource failed., nvinfer error:NVDSINFER_CONFIG_FAILED Error: gst-resource-error-quark: Failed to create NvDsInferContext instance (1): gstnvinfer.cpp(846): gst_nvinfer_start (): /GstPipeline:pipeline0/GstNvInfer:primary-inference: Config file path: dstest1_pgie_config.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED`

I would like to know if I'm doing something wrong or if there is anything wrong in the repo. Happy to give any information needed!

marcoslucianops commented 2 years ago

Your dstest1_pgie_config.txt file is wrong

madisi98 commented 2 years ago

I just copied the one provided by the repo and updated the paths to match my installation. Not sure where I went wrong. Here it is:

[property] gpu-id=0 net-scale-factor=0.0039215697906911373 model-color-format=0 custom-network-config=/workspace/packages/yolor/yolor/cfg/yolor_p6.cfg model-file=/workspace/packages/yolor/yolor/yolor_p6.wts model-engine-file=/workspace/packages/yolor/yolor/model.prueba-batch1-32.engine

int8-calib-file=calib.table

labelfile-path=/model/detect/coco.names batch-size=1 network-mode=0 num-detected-classes=80 interval=0 gie-unique-id=1 process-mode=1 network-type=0 cluster-mode=2 maintain-aspect-ratio=0 parse-bbox-func-name=NvDsInferParseYolo custom-lib-path=/workspace/packages/DeepStream-Yolo/nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so engine-create-func-name=NvDsInferYoloCudaEngineGet

[class-attrs-all] nms-iou-threshold=0.45 pre-cluster-threshold=0.25

marcoslucianops commented 2 years ago 
ERROR: nvdsinfer_context_impl.cpp:589 parse label file:/model failed, nvinfer error:NVDSINFER_CONFIG_FAILED
labelfile-path=/model/detect/coco.names

Try to use the relative path for labelfile-path


model-engine-file=/workspace/packages/yolor/yolor/model.prueba-batch1-32.engine

The engine file will be generated in the same directory that you are running the deepstream command with model_b1_gpu0_fp32.engine filename (b1: batch-size=1 and fp32: network-mode=0).

madisi98 commented 2 years ago

It works now, thank you so much !!