YOLO NAS'S Precision is significantly lower compare to other later YOLO model even when using same dataset ?

[ikmal-asraf]

💡 Your Question

I was doing a object detection project , which is a drowning detection . Below is the result of YOLO-NAS S with 250 epochs.

Despite having high recall and mAP value , both precision and f1 score is low under (0.1) . When testing the model , it do quite well on detecting class despite having low precision .

So I try to find if other people face the same issue . 1.https://github.com/Deci-AI/super-gradients/issues/1191 2.https://github.com/Deci-AI/super-gradients/issues/1734 3.Performance Analysis of YOLO-NAS SOTA Models on CAL Tool Detection YoloNASCALPaper-3.pdf

4.ASSESSING THE EFFECTIVENESS OF YOLO ARCHITECTURES FOR SMOKE AND WILDFIRE DETECTION Assessing_the_Effectiveness_of_YOLO_Architectures_for_Smoke_and_Wildfire_Detection.pdf

5.COMPARATIVE STUDY OF YOLOV8 AND YOLO-NAS FOR AGRICULTURE APPLICATION Comparative_study_of_YOLOv8_and_YOLO-NAS_for_agriculture_application.pdf

From the 5 cases above , it seem like having very low precision but at the same time high recall and mAP is quite normal for YOLO-NAS. When compare with other YOLO version with same dataset, the other YOLO model does not face the same problem . Why does this happen , considering that YOLO NAS generally perform better than other lower YOLO model? Is it because how different YOLO-NAS been evaluate ?

Versions

No response

[YGBRS]

Not an expert at all but how did you train your model ? I think (if you follow most tutorials), there is several data augmentations step performed by default (pretty hidden) on the training dataset. That was messing up my training a lot, especially the mixup one.

[ikmal-asraf]

@YGBRS , I use Roboflow to store and data augmentations , to train I use Google Colab notebook from here https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/train-yolo-nas-on-custom-dataset.ipynb#scrollTo=SdQ5JGblbJTk .

[YGBRS]

You can see the list of default data transforms done on your training data with train_data.dataset.transforms If you want to remove some transforms that may or may not mess up your training you can remove them with train_data.dataset.transforms.pop(index). You can plot a sample of your training data with the transforms with train_data.dataset.plot() If you see the Mixup one , try to remove it and retrain your model. Just test whatever you want from there.

[CrasCris]

It depends if you see the COCO dataset the average of the images have a pixel rate, if you are training to train imagen that the size are too different the results could change, so maybe if you resized your dataset would help