dusty-nv / jetson-inference

Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
https://developer.nvidia.com/embedded/twodaystoademo
MIT License
7.78k stars 2.98k forks source link

Retrained ssd-mobilenet-v1 can't correctly detect objects with a custom parser in deepstream #1309

Closed shaojun closed 2 years ago

shaojun commented 2 years ago

Thanks for the great repo, dusty.

I've re-trained a ssd-mobilenet-V1 model by code from the pytorch-ssd on PASCAL-VOC dataset, after convert it to .onnx, it can correctly run with dectections at Jetson Nano 4G based your code via:

root@shao-jsn-4g:/jetson-inference/python/training/detection/ssd# detectnet --model=models/GPU_trained_Models/ssd_epoch9.onnx --labels=models/GPU_trained_Models/voc-model-labels.txt --input-blob=input_0 --output-cvg=scores --output-bbox=boxes models/GPU_trained_Models/sample_720p.mp4

Then, I'm trying to run this model via DeepStream 6 (at the same hardware), I see one of you previous post say that a Custom Parser is needed for deepstream, and can refer: https://elinux.org/index.php?title=Jetson/L4T/TRT_Customized_Example#Custom_Parser_for_SSD-MobileNet_Trained_by_Jetson-inference to do it, I've followed the steps and build it, added the below content to a deepstream app config:

[property] workspace-size=600 gpu-id=0 net-scale-factor=0.003921569790691137 onnx-file=/home/shao/jetson-inference/python/training/detection/ssd/models/GPU_trained_Models/ssd_epoch9.onnx labelfile-path=/home/shao/jetson-inference/python/training/detection/ssd/models/GPU_trained_Models/voc-model-labels.txt batch-size=3 model-color-format=0 network-mode=2 num-detected-classes=21 gie-unique-id=1 output-blob-names=boxes;scores parse-bbox-func-name=NvDsInferParseCustomSSD custom-lib-path=/opt/nvidia/deepstream/deepstream/sources/objectDetector_SSD/nvdsinfer_custom_impl_ssd/libnvdsinfer_custom_impl_ssd.so

[class-attrs-all]

pre-cluster-threshold=0.5

eps=0.2

group-threshold=1

Use the config params below for NMS clustering mode

[class-attrs-all] topk=20 nms-iou-threshold=0.5 pre-cluster-threshold=0.2

but after run deepstream-app -c aboveConfig.txt, the bounding box and detection seems all incorrect: with the video playing, the whole screen marked with a bounding box with label bicycle(though only first several frames have a man walked with a bicycle), and right top corner show some strange bounding boxes.

image

the source for Custom Parser and .so files are:

nvdsinfer_custom_impl_ssd.zip

BTW, there's a build-in ssd custom parser in deepstream 6 sample app (as you can see the default config point to it already custom-lib-path=/opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_infercustomparser.so)then why we need compile one here?

Thanks.

dusty-nv commented 2 years ago

Hi @shaojun, unfortunately I'm not familiar with the DeepStream side of things, and my colleague @aastanv had created that wiki page with the custom parser. I would recommend posting to the DeepStream SDK forum so that our DeepStream experts can take a look:

https://forums.developer.nvidia.com/c/accelerated-computing/intelligent-video-analytics/deepstream-sdk/15