Use 2 weights with different num classes

[avighi]

I would like to configure as primary-gie 1 model with one class, and inside the bounding box look for other objects using another model (secondary-gie0) with 35 classes.

I'm getting this error: Num classes mismatch. Configured: 35, detected by network: 1

The cfg and config files are configured correctly. Is this the right way? I'm doing well? Thanks

[marcoslucianops]

If you are using YOLO as secondary-gie, you need to change the sgie yoloPlugins.h file

const char* YOLOLAYER_PLUGIN_VERSION {"1"};

to

const char* YOLOLAYER_PLUGIN_VERSION {"2"};

Compile and generate the engine again.

You need to clone this repo two times, changing the folder names to pgie and sgie. In the pgie folder, you should keep the version = 1. In the sgie folder, you must change to version = 2.

You also need to change custom-network-config key (in each config_infer.txt file)

pgie example

custom-network-config=pgie/yolov4.cfg

sgie example

custom-network-config=sgie/yolov5s.cfg

The engines will be gerenerate in the path you are running the command, so you need to move the generated engine to the pgie/sgie folder. The sgie engine is built first, and then the pgie engine.

I will write a doc in the next days with multiple gie implementation.

[avighi]

Okay thank you, I kindly wait for the document with the detailed guide.

[avighi]

Sorry, one thing is not clear to me: is it necessary to have inside the parent folder "deepstream-yolo", 2 new folders pgie and sgie right? And then how should I start? Sorry but I am not familiar with deepstream with yolo yet.

I receive this error: Error Code 1: Serialization (Serialization assertion creator failed.Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry)

Thanks

[marcoslucianops]

https://github.com/marcoslucianops/DeepStream-Yolo/blob/master/docs/multipleGIEs.md

[avighi]

Ok thanks, but with Jetson nano I receive this error during detections (2 gie):

CUDA failure: too many resources requested for launch in file yoloPlugins.cpp What could be the problem?

And also i noticed that only gie1 model detects objects, gie2 seems not to work

[marcoslucianops]

Can you send me your edited config_infer_primary and the deepstream_app_config.txt files?

[avighi]

I send you an email now. Thanks

[marcoslucianops]

What's the topK value in the config_nms.txt?

[avighi]

topk=300 for both of them (pgie and sgie)

Is right?

[marcoslucianops]

Can you test with the new commit files?

[avighi]

Yes, I receive this error when I try to load my second weights file.

Number of unused weights left: 1 deepstream-app: yolo.cpp:438: NvDsInferStatus Yolo::buildYoloNetwork(std::vector&, nvinfer1::INetworkDefinition&): Assertion `0' failed.

I share with you. On Darknet it loads corrently. Thanks gie2.zip

[marcoslucianops]

Please change the yolov2_ocr-net.cfg to yolov2-tiny_ocr-net.cfg.

[avighi]

Ok, building TensorRT engine now It works but I cannot detect objects within the detection from the first model using second model.

I'll try to explain it better: The first model detects the class "License plate" (it works) then I would like to search for letters within the first detection using the second model (I don't see the detections). I am sure it works because if I use darknet by passing it a license plate, the model returns the letter detections.

The second model is applied to the first detection as if it were a crop of the sub area right?

[marcoslucianops]

Yes, the second model runs on the detected area of the first model.

Can you send a video sample to run the model and test?

[avighi]

For privacy I can't send here a video with Licence plate of vehicles, but I can confirm you that if you load this model: https://github.com/marcoslucianops/DeepStream-Yolo/files/9196049/gie2.zip in Darknet (for example) and you use a video with license plates, it works.

Instead, if I use both of models (licence plate and next "https://github.com/marcoslucianops/DeepStream-Yolo/files/9196049/gie2.zip" for OCR) you can see only the licence plate detection, not detections of single letters in the licence plate.

If you can, we could use skype or other methods for sharing.

Thanks

[avighi]

You can use any video with cars and licence plate

[marcoslucianops]

I'm geting the outputs with your model as secondary detection.

[avighi]

Can you share your repo? i am using jetson nano, maybe it is a ram or device problem? Because secondary detection doesn't works to me. I'm using DS 6.0 but in effect I receive this error:

**PERF: 0.00 (0.00)
CUDA failure: too many resources requested for launch in file yoloPlugins.cpp at line 253 Aborted (core dumped)

But if I use Jetson Xavier, It works.

Thanks

[marcoslucianops]

Can you test with the new nvdsinfer_custom_impl_Yolo from the current repo branch? Does the secondary detection work on NX?

[avighi]

With new "nvdsinfer_custom_impl_Yolo":

Building YOLO network complete Building the TensorRT Engine

Killed

On Jetson AGX Xavier the secondary detection works, I don't have an NX module

[avighi]

Update: "CUDA failure: too many resources requested for launch in file yoloPlugins.cpp at line 253"

Same problem when jetson nano detects the object of class in first GIE also with new repo.

Thanks

[marcoslucianops]

I brought a Jetson Nano from my trip. I will try to reproduce this issue.