YoloNAS no detections after conversion

[whittenator]

To recreate the issue:

Step 1: Trained Yolo NAS model and got a ckpt_best.pth model file. (Tested this model on several videos and it works perfectly). Step 2: Followed the steps outlined in the README to convert to ONNX. I trained a custom model with 2 classes only and I trained it with the large yolo nas model, so I ran the following command to convert my model: python3 export_yolonas.py -m yolo_nas_l -w yolo_nas_l.pth --dynamic --classes 2 Step 3: I ran the deepstream-app with the proper config file pointing to the plugin. It successfully converted the .onnx file converted from above to a .engine file and started running on the video. However, I'm not seeing any bounding boxes on the video at all. I tried converting to FP16, FP32 and INT8(All of which I have done with success with YoloV7).

Please see attach my config files(config_infer_primary_yolo-nas.txt and deepstreawm_app_config_test.txt) for reference: config_infer_primary_yolo-nas.txt deepstream_app_config_test.txt

Any ideas?

[marcoslucianops]

Can you send the model to my email for testing?

[marcoslucianops]

I added the config_infer_primary_yolonas_custom.txt file to the repo. For custom YOLO-NAS model you should use it.

The pretrained YOLO-NAS is using net-scale-factor=0.0039215697906911373 and the custom YOLO-NAS is trained with net-scale-factor=1.

[whittenator]

Thank you!

How can I determine the net-scale-factor of a particular model? For example, if I convert the model to use INT8, then the detections disappear and I used 2,000 images from validation to run the calibrations.

[marcoslucianops]

The net-scale-factor and offsets keys are related to image normalization in the image preprocessing.

The net-scale-factor is related to std values (example for std = 0.229, 0.224, 0.225:

net_scale_fator = 1 / (np.array([0.229, 0.224, 0.225]).mean() * 255)
print(net_scale_fator)

The offsets is related to mean values (example for mean = 0.485, 0.456, 0.406:

offsets = np.array([0.485, 0.456, 0.406]) * 255
print(";".join(str(x) for x in offsets))

About the INT8 model, I need to check later how to export it correctly to use in the DeepStream. For now it should not work.

[naseemap-er]

Hi @marcoslucianops , I trained my custom YOLO-NAS model using https://github.com/naseemap47/YOLO-NAS where I will find my model std I am not giving any std input value. I only given input size as 640x640

[marcoslucianops]

@naseemap-er try both config files available for YOLO-NAS and check which one you will get detections.

[naseemap-er]

@marcoslucianops When I used coco net-scale-factor, its working for me. But I am getting few false detections on the sides of frame. But when I use YOLO-NAS inference. I didn't get that kind of detections. When I try with net-scale-factor = 1. I am getting full false predictions. So I thinking it's due to net-scale-factor. But to find my net-scale-factor, I need std, according your formula. So can you help me to find the std value from training.

[marcoslucianops]

If you didn't change it in the training, it should be the default values.

[naseemap-er]

@marcoslucianops thanks for your support. Can you tell me when will be the Deepstream YOLOv9 will release ? It will very helpful.