FPS..

[ONNONS]

I've tried deleting and installing, deleting and repeating dozens of times, but fps is 2-3... plz help me

~/Desktop/DeepStream-Yolo-master$ deepstream-app -c deepstream_app_config.txt

Using winsys: x11 ERROR: Deserialize engine failed because file path: /home/iisl/Desktop/DeepStream-Yolo-master/model_b1_gpu0_fp32.engine open error 0:00:06.930669600 21543 0xdaf8200 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() [UID = 1]: deserialize engine from file :/home/iisl/Desktop/DeepStream-Yolo-master/model_b1_gpu0_fp32.engine failed 0:00:06.943817699 21543 0xdaf8200 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() [UID = 1]: deserialize backend context from engine from file :/home/iisl/Desktop/DeepStream-Yolo-master/model_b1_gpu0_fp32.engine failed, try rebuild 0:00:06.943875878 21543 0xdaf8200 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() [UID = 1]: Trying to create engine from model files

Loading pre-trained weights Loading weights of yolov5n complete Total weights read: 1881661 Building YOLO network

  layer                        input               output         weightPtr

(0) conv_silu 3 x 640 x 640 16 x 320 x 320 1792
(1) conv_silu 16 x 320 x 320 32 x 160 x 160 6528
(2) conv_silu 32 x 160 x 160 16 x 160 x 160 7104
(3) route - 32 x 160 x 160 7104
(4) conv_silu 32 x 160 x 160 16 x 160 x 160 7680
(5) conv_silu 16 x 160 x 160 16 x 160 x 160 8000
(6) conv_silu 16 x 160 x 160 16 x 160 x 160 10368
(7) shortcut_linear: 4 - 16 x 160 x 160 -
(8) route - 32 x 160 x 160 10368
(9) conv_silu 32 x 160 x 160 32 x 160 x 160 11520
(10) conv_silu 32 x 160 x 160 64 x 80 x 80 30208
(11) conv_silu 64 x 80 x 80 32 x 80 x 80 32384
(12) route - 64 x 80 x 80 32384
(13) conv_silu 64 x 80 x 80 32 x 80 x 80 34560
(14) conv_silu 32 x 80 x 80 32 x 80 x 80 35712
(15) conv_silu 32 x 80 x 80 32 x 80 x 80 45056
(16) shortcut_linear: 13 - 32 x 80 x 80 -
(17) conv_silu 32 x 80 x 80 32 x 80 x 80 46208
(18) conv_silu 32 x 80 x 80 32 x 80 x 80 55552
(19) shortcut_linear: 16 - 32 x 80 x 80 -
(20) route - 64 x 80 x 80 55552
(21) conv_silu 64 x 80 x 80 64 x 80 x 80 59904
(22) conv_silu 64 x 80 x 80 128 x 40 x 40 134144 (23) conv_silu 128 x 40 x 40 64 x 40 x 40 142592 (24) route - 128 x 40 x 40 142592 (25) conv_silu 128 x 40 x 40 64 x 40 x 40 151040 (26) conv_silu 64 x 40 x 40 64 x 40 x 40 155392 (27) conv_silu 64 x 40 x 40 64 x 40 x 40 192512 (28) shortcut_linear: 25 - 64 x 40 x 40 -
(29) conv_silu 64 x 40 x 40 64 x 40 x 40 196864 (30) conv_silu 64 x 40 x 40 64 x 40 x 40 233984 (31) shortcut_linear: 28 - 64 x 40 x 40 -
(32) conv_silu 64 x 40 x 40 64 x 40 x 40 238336 (33) conv_silu 64 x 40 x 40 64 x 40 x 40 275456 (34) shortcut_linear: 31 - 64 x 40 x 40 -
(35) route - 128 x 40 x 40 275456 (36) conv_silu 128 x 40 x 40 128 x 40 x 40 292352 (37) conv_silu 128 x 40 x 40 256 x 20 x 20 588288 (38) conv_silu 256 x 20 x 20 128 x 20 x 20 621568 (39) route - 256 x 20 x 20 621568 (40) conv_silu 256 x 20 x 20 128 x 20 x 20 654848 (41) conv_silu 128 x 20 x 20 128 x 20 x 20 671744 (42) conv_silu 128 x 20 x 20 128 x 20 x 20 819712 (43) shortcut_linear: 40 - 128 x 20 x 20 -
(44) route - 256 x 20 x 20 819712 (45) conv_silu 256 x 20 x 20 256 x 20 x 20 886272 (46) conv_silu 256 x 20 x 20 128 x 20 x 20 919552 (47) maxpool 128 x 20 x 20 128 x 20 x 20 919552 (48) route - 128 x 20 x 20 919552 (49) maxpool 128 x 20 x 20 128 x 20 x 20 919552 (50) route - 128 x 20 x 20 919552 (51) maxpool 128 x 20 x 20 128 x 20 x 20 919552 (52) route - 512 x 20 x 20 919552 (53) conv_silu 512 x 20 x 20 256 x 20 x 20 1051648 (54) conv_silu 256 x 20 x 20 128 x 20 x 20 1084928 (55) upsample 128 x 20 x 20 128 x 40 x 40 -
(56) route - 256 x 40 x 40 1084928 (57) conv_silu 256 x 40 x 40 64 x 40 x 40 1101568 (58) route - 256 x 40 x 40 1101568 (59) conv_silu 256 x 40 x 40 64 x 40 x 40 1118208 (60) conv_silu 64 x 40 x 40 64 x 40 x 40 1122560 (61) conv_silu 64 x 40 x 40 64 x 40 x 40 1159680 (62) route - 128 x 40 x 40 1159680 (63) conv_silu 128 x 40 x 40 128 x 40 x 40 1176576 (64) conv_silu 128 x 40 x 40 64 x 40 x 40 1185024 (65) upsample 64 x 40 x 40 64 x 80 x 80 -
(66) route - 128 x 80 x 80 1185024 (67) conv_silu 128 x 80 x 80 32 x 80 x 80 1189248 (68) route - 128 x 80 x 80 1189248 (69) conv_silu 128 x 80 x 80 32 x 80 x 80 1193472 (70) conv_silu 32 x 80 x 80 32 x 80 x 80 1194624 (71) conv_silu 32 x 80 x 80 32 x 80 x 80 1203968 (72) route - 64 x 80 x 80 1203968 (73) conv_silu 64 x 80 x 80 64 x 80 x 80 1208320 (74) conv_silu 64 x 80 x 80 64 x 40 x 40 1245440 (75) route - 128 x 40 x 40 1245440 (76) conv_silu 128 x 40 x 40 64 x 40 x 40 1253888 (77) route - 128 x 40 x 40 1253888 (78) conv_silu 128 x 40 x 40 64 x 40 x 40 1262336 (79) conv_silu 64 x 40 x 40 64 x 40 x 40 1266688 (80) conv_silu 64 x 40 x 40 64 x 40 x 40 1303808 (81) route - 128 x 40 x 40 1303808 (82) conv_silu 128 x 40 x 40 128 x 40 x 40 1320704 (83) conv_silu 128 x 40 x 40 128 x 20 x 20 1468672 (84) route - 256 x 20 x 20 1468672 (85) conv_silu 256 x 20 x 20 128 x 20 x 20 1501952 (86) route - 256 x 20 x 20 1501952 (87) conv_silu 256 x 20 x 20 128 x 20 x 20 1535232 (88) conv_silu 128 x 20 x 20 128 x 20 x 20 1552128 (89) conv_silu 128 x 20 x 20 128 x 20 x 20 1700096 (90) route - 256 x 20 x 20 1700096 (91) conv_silu 256 x 20 x 20 256 x 20 x 20 1766656 (92) route - 64 x 80 x 80 1766656 (93) conv_logistic 64 x 80 x 80 255 x 80 x 80 1783231 (94) yolo 255 x 80 x 80 255 x 80 x 80 1783231 (95) route - 128 x 40 x 40 1783231 (96) conv_logistic 128 x 40 x 40 255 x 40 x 40 1816126 (97) yolo 255 x 40 x 40 255 x 40 x 40 1816126 (98) route - 256 x 20 x 20 1816126 (99) conv_logistic 256 x 20 x 20 255 x 20 x 20 1881661 (100) yolo 255 x 20 x 20 255 x 20 x 20 1881661 Output YOLO blob names: yolo_95 yolo_98 yolo_101 Total number of YOLO layers: 275 Building YOLO network complete Building the TensorRT Engine WARNING: [TRT]: Detected invalid timing cache, setup a local cache instead Building complete

0:02:48.032024989 21543 0xdaf8200 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() [UID = 1]: serialize cuda engine to file: /home/iisl/Desktop/DeepStream-Yolo-master/model_b1_gpu0_fp32.engine successfully INFO: [Implicit Engine Info]: layers num: 4 0 INPUT kFLOAT data 3x640x640
1 OUTPUT kFLOAT yolo_95 255x80x80
2 OUTPUT kFLOAT yolo_98 255x40x40
3 OUTPUT kFLOAT yolo_101 255x20x20

0:02:48.690384379 21543 0xdaf8200 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus: [UID 1]: Load new model:/home/iisl/Desktop/DeepStream-Yolo-master/config_infer_primary_yoloV5.txt sucessfully

Runtime commands: h: Print this help q: Quit

p: Pause
r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source. To go back to the tiled display, right-click anywhere on the window.

PERF: FPS 0 (Avg)
PERF: 0.00 (0.00)
** INFO: : Pipeline ready

Opening in BLOCKING MODE NvMMLiteOpen : Block : BlockType = 261 NVMEDIA: Reading vendor.tegra.display-size : status: 6 NvMMLiteBlockCreate : Block : BlockType = 261 ** INFO: : Pipeline running

PERF: 0.00 (0.00)
PERF: 0.00 (0.00)
PERF: 2.40 (2.26)
PERF: 2.81 (1.11)
PERF: 0.00 (0.71)
PERF: 0.00 (0.52)
PERF: 0.00 (0.41)
PERF: 0.00 (0.34)
**PERF: 0.00 (0.29)
Quitting App run successful

[ONNONS]

It may not be possible to load the engine file. I followed your tutorial as it is, extracted the cfg and wts files with yolov5n.pt and created the engine file at the same time as running it from the Deepstream-Yolo file... Please help me

[marcoslucianops]

https://github.com/marcoslucianops/DeepStream-Yolo/issues/110

[marcoslucianops]

I'm working in performance and accuracy optimization, it will be available soon.

[marcoslucianops]

It may not be possible to load the engine file. I followed your tutorial as it is, extracted the cfg and wts files with yolov5n.pt and created the engine file at the same time as running it from the Deepstream-Yolo file... Please help me

Did you change batch-size or network-mode in config_infer_primary_yoloV5.txt file?

[ONNONS]

No, I didn't modify anything and used it as it is.

[marcoslucianops]

Do you mean that the engine is created every time you run the deepstream-app command?

[ONNONS]

The engine file is formed on the first run, and it is no longer created on the next run. The standard of yolov5n.

[ONNONS]

It was executed successfully without an error message. But fps is low.

Should I use the triton server? I don't know what the problem is...

[zetxy]

I believe that you are running the model on Jetson Nano. From my experience this board is not particularly powerful. So running a model such as Yolov5 on this board is likely to achieve pretty low FPS.

I would suggest checking the following :

1. Check if the power supply to the device is sufficient. (I believe you want 20W - please double check this before blowing up your board)
2. Set the sink type to FAKE, do not plot anything on display.
3. Is the model properly quantized.
4. What is the load of the unit during processing? Is it low or is there something near 100%?

Why do you think Triton server would make a difference.

Maybe providing more information about your problem would enhance the chances of other participants helping you

[marcoslucianops]

About the performance and accuracy of YOLOv5 and YOLOR, I'm checking and I will have a reply soon.

[marcoslucianops]

Repo updated and fixed the performance (FP16 mode) and accuracy of YOLOv5 models. Please use the new file to convert the model and check it again.