Missing detection in some frames on two videos

[tuneshverma]

I am using deepstream pipeline (custom deepstream-test3) for detection. I generated the ".so" file.

When I run it on one video there are no missing detections. But while running on two videos, there are missing detections in a few frames. While running on two same videos also see the same issue and also miss on a different frame, there will not be detected in both videos at the same time. It seems the detection just switches from one frame to another in case of multi-streaming and skipping the frame to detect. Link for deepstream file. https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/blob/master/apps/deepstream-test3/deepstream_test_3.py

[marcoslucianops]

Are you using interval > 0 in the config_infer_primary file?

[tuneshverma]

No, it's interval = 0.

[marcoslucianops]

I can't reproduce this issue. Can you send more details?

[tuneshverma]

Maybe with exactly my config and deepstream file, you can look into it further.

Here is the link for the config file I am using : https://www.dropbox.com/s/f3i8jrzey9yoay9/config_infer_primary_yolov4-tiny.txt?dl=0

Here is the link for the deepstream-test3 code: https://www.dropbox.com/s/hp7q2hu29lkynkr/deepstream_test_3_working.py?dl=0

[marcoslucianops]

I did some changes in the repo. Please test with the new files.

There are many incorrect lines in your config_infer_primary_yolov4-tiny.txt file. Please use this config_infer_primary.txt as reference.

[tuneshverma]

I re-cloned the repo and make the .so file again. But now it can't find cfg for for model. I gave absolute path for all, then also same error

WARNING: ../nvdsinfer/nvdsinfer_model_builder.cpp:1482 Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-6.1/sources/DeepStream-Yolo/model_b1_gpu0_fp32.engine open error 0:00:01.518589132 2217 0x56276197d610 WARN nvinfer gstnvinfer.cpp:656:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() [UID = 1]: deserialize engine from file :/opt/nvidia/deepstream/deepstream-6.1/sources/DeepStream-Yolo/model_b1_gpu0_fp32.engine failed 0:00:01.519245800 2217 0x56276197d610 WARN nvinfer gstnvinfer.cpp:656:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() [UID = 1]: deserialize backend context from engine from file :/opt/nvidia/deepstream/deepstream-6.1/sources/DeepStream-Yolo/model_b1_gpu0_fp32.engine failed, try rebuild 0:00:01.519265699 2217 0x56276197d610 INFO nvinfer gstnvinfer.cpp:659:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() [UID = 1]: Trying to create engine from model files

File does not exist: /opt/nvidia/deepstream/deepstream-6.1/sources/DeepStream-Yolo/yolov4.cfg YOLO config file or weights file is not exist

[tuneshverma]

When I am using the old config with the new ".so" file then this error is coming.

0:00:33.344393676 13982 0x7f13140022f0 INFO nvinfer gstnvinfer_impl.cpp:328:notifyLoadModelStatus: [UID 2]: Load new model:test_classifier_config_dli.txt sucessfully Deserialize yoloLayer plugin: yolo Segmentation fault (core dumped)

[marcoslucianops]

You need to delete the old yolo engine file, compile the new so file, change the custom-network-config and model-file path in ther new config_infer file, and run the code again.

[tuneshverma]

When I deleted the engine files. Number of unused weights left: 18446744073709358080 python3: yolo.cpp:427: NvDsInferStatus Yolo::buildYoloNetwork(std::vector&, nvinfer1::INetworkDefinition&): Assertion `0' failed. Aborted (core dumped)

[marcoslucianops]

Which model you are using and which is the custom-network-config and model-file paths you put?

[tuneshverma]

Which model you are using and which is the custom-network-config and model-file paths you put?

I am using tiny yolov4 model trained on our own dataset. tiny_yolov4_3l.cfg and weights according to it.

[tuneshverma]

I removed the repo and cloned it again, this time is saved the cfg and weights in the directory before making the so file. I did not get the file does not exist error.

But I got this error.

Loading pre-trained weights Loading weights of hand_yolov4-tiny-3l complete Total weights read: 6120982 Building YOLO network

       Layer                         Input Shape         Output Shape        WeightPtr

(0) conv_leaky [3, 608, 608] [32, 304, 304] 992 (1) conv_leaky [32, 304, 304] [64, 152, 152] 19680 (2) conv_leaky [64, 152, 152] [64, 152, 152] 56800 (3) route: 2 - [64, 152, 152] - (4) conv_leaky [64, 152, 152] [32, 152, 152] 75360 (5) conv_leaky [32, 152, 152] [32, 152, 152] 84704 (6) route: 5, 4 - [64, 152, 152] - (7) conv_leaky [64, 152, 152] [64, 152, 152] 89056 (8) route: 2, 7 - [128, 152, 152] - (9) maxpool [128, 152, 152] [128, 76, 76] - (10) conv_leaky [128, 76, 76] [128, 76, 76] 237024 (11) route: 10 - [128, 76, 76] - (12) conv_leaky [128, 76, 76] [64, 76, 76] 311008 (13) conv_leaky [64, 76, 76] [64, 76, 76] 348128 (14) route: 13, 12 - [128, 76, 76] - (15) conv_leaky [128, 76, 76] [128, 76, 76] 365024 (16) route: 10, 15 - [256, 76, 76] - (17) maxpool [256, 76, 76] [256, 38, 38] - (18) conv_leaky [256, 38, 38] [256, 38, 38] 955872 (19) route: 18 - [256, 38, 38] - (20) conv_leaky [256, 38, 38] [128, 38, 38] 1251296 (21) conv_leaky [128, 38, 38] [128, 38, 38] 1399264 (22) route: 21, 20 - [256, 38, 38] - (23) conv_leaky [256, 38, 38] [256, 38, 38] 1465824 (24) route: 18, 23 - [512, 38, 38] - (25) maxpool [512, 38, 38] [512, 19, 19] - (26) conv_leaky [512, 19, 19] [512, 19, 19] 3827168 (27) conv_leaky [512, 19, 19] [256, 19, 19] 3959264 (28) conv_leaky [256, 19, 19] [512, 19, 19] 5140960 (29) conv_linear [512, 19, 19] [18, 19, 19] 5150194 (30) yolo [18, 19, 19] - - (31) route: 27 - [256, 19, 19] - (32) conv_leaky [256, 19, 19] [128, 19, 19] 5183474 (33) upsample [128, 19, 19] [128, 38, 38] - (34) route: 33, 23 - [384, 38, 38] - (35) conv_leaky [384, 38, 38] [256, 38, 38] 6069234 (36) conv_linear [256, 38, 38] [18, 38, 38] 6073860 (37) yolo [18, 38, 38] - - (38) route: 35 - [256, 38, 38] - (39) conv_leaky [256, 38, 38] [64, 38, 38] 6090500 (40) upsample [64, 38, 38] [64, 76, 76] - (41) route: 40, 15 - [192, 76, 76] - (42) conv_leaky [192, 76, 76] [128, 76, 76] 6312196 (43) conv_linear [128, 76, 76] [18, 76, 76] 6314518 (44) yolo [18, 76, 76] - -

Number of unused weights left: 18446744073709358080 deepstream-app: yolo.cpp:427: NvDsInferStatus Yolo::buildYoloNetwork(std::vector&, nvinfer1::INetworkDefinition&): Assertion `0' failed. Aborted (core dumped)

[marcoslucianops]

I think you are using the wrong weights file for this cfg.

[tuneshverma]

Last time, when I did the conversion by using your code, I used the same cfg and weights file. It worked perfectly. It was just that on two videos I was facing issues.

[marcoslucianops]

I can reproduce the issue, working on it.

[marcoslucianops]

Fixed, please check with the new files.

[tuneshverma]

Yes, it fixed the problem, and it's also working perfectly fine with two videos. Thank you very much.