Low Precision while training

[lijo381]

Training a yolonas model on satellite image dataset, the mAP values and recall improve as expected. But the precision is very low because of which inference is very poor. Loss used is ppyoloeloss with SGD optimizer which internally uses focal loss, but looks like it is not helping the precision. Is there any way to improve the precision from training side ?

[BloodAxe]

Can you provide more information about your experiment? 1) Do you use pre-trained weights of yolo-nas? 2) Can you share the train/validation loss & mAP plots? 3) What are the size of objects in your dataset? Maybe they are too small? 4) Does the same behaviour holds when using Adam?

[lijo381]

Do you use pre-trained weights of yolo-nas?

yes i used the coco weights

Can you share the train/validation loss & mAP plots?

Shared as attachments below

What are the size of objects in your dataset? Maybe they are too small?

The objects are pretty small as well there is imbalance in dataset. My assumption was focal loss will take care of imbalance.

Does the same behaviour holds when using Adam?

Yes the behaviour is same with adam, though with SGD i am seeing faster convergence.

[BloodAxe]

This clearly looks like an overfit. Neither focal loss nor certain architecture can solve this magically. You can also try adding DetectionMixup augmentation which usually helps.

On RF-100 we observe this is important not to filter any boxes even small once: https://github.com/Deci-AI/super-gradients/blob/master/src/super_gradients/recipes/dataset_params/roboflow_detection_dataset_params.yaml#L23 You may also want to use this trick in your training.

[lijo381]

You can also try adding DetectionMixup augmentation which usually helps.

Does this needs to be implemented seperately or is just a case of enabling ? If its the latter could you please point to it.

Neither focal loss nor certain architecture can solve this magically.

I was able to generalise the yolov5 model for the same dataset with focal loss.

not to filter any boxes even small once

will give it a try

[utkutpcgl]

I have also faced a similar issue, precision is very low. My dataset is not so small and I use Adam. I might try this filter_box_candidates=False also.

[ShpihanVlad]

Hi, @lijo381 , @utkutpcgl , were your attempts successfull? Thx for your time

[utkutpcgl]

Not really, I had better results with yolov5, I might be missing something.

[ShpihanVlad]

Hi @utkutpcgl , in case your mAP scores were good enough, but precision is so small, #1305 may help you. For comparison, YOLOv5 calculates P and R on optimal confidence threshold, found by F1-curve

[BloodAxe]

Since 3.6 release we have enabled optimal confidence threshold computation by default in our DetectionMetrics so it should be higher now.