Print test metrics

[tjasmin111]

💡 Your Question

I want to test the trained yolo nas model, but when running it like below, it doesn't print anything, no metrics. How can I enable printing all those metrics here?

best_model = models.get(
    MODEL_ARCH,
    num_classes=len(dataset_params['classes']),
    checkpoint_path=f"{CHECKPOINT_DIR}/{EXPERIMENT_NAME}/RUN_20231019_025212_979921/average_model.pth"
).to(DEVICE)

#evaluate model

trainer.test(
    model=best_model,
    test_loader=test_data,
    test_metrics_list=DetectionMetrics_050(
        score_thres=0.1,
        top_k_predictions=300,
        num_cls=len(dataset_params['classes']),
        normalize_targets=True,
        post_prediction_callback=PPYoloEPostPredictionCallback(
            score_threshold=0.01,
            nms_top_k=1000,
            max_predictions=300,
            nms_threshold=0.7
        )
    )
)

Versions

No response

[BloodAxe]

SG version? Colab env or script run?

[tjasmin111]

Yes from super gradients. It's a python script.

[BloodAxe]

A trainer.test call does not print metrics. It returns a dictionary of metrics: test_metrics = trainer.test(...). So if you want to see then you have to print them manually:

test_metrics = trainer.test(...)
print(test_metrics)

[tjasmin111]

Thanks. But When I print it, it just prints Recall and mAP and such for ALL classes. I want a more detailed one (just like YOLO) reporting these for each class. Is there a way to do that?

[BloodAxe]

Can you provide an example of expected output?

[tjasmin111]

Something like this, per class:

             Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████
               all         468       1128      0.789       0.75      0.816      0.632
             Object1        468        345      0.845      0.836      0.869      0.763
              Object2       468        743      0.733      0.664      0.762      0.501

Speed: 0.3ms preprocess, 1.6ms inference, 0.0ms loss, 0.5ms postprocess per image

[CodingMonkey3688]

Add the include_classwise_ap=True in the DetectionMetrics_050

[tjasmin111]

Thanks. It only adds the AP@0.50 for classes, not Precision, F1, etc. It would benefit from a more detailed list, as a feature request. Also, it says class0, class1! Would be better to show the actual name of class.