COCO Recipe reporting low precision

[thomasmilas-swe-bc]

💡 Your Question

I'm evaluating supergradients and YoloNAS and I'm trying to use the coco recipe to train a model from scratch. I am getting< .20onPrecision@0.50:0.95`. For an out of the box recipe I assumed the numbers would be more like a expect, getting a closer match to recall. Recall raises as expected, as well does MAP.

On other builds (fine tuning with my own data) I see similar issues with the precision metrics. when I run an external evaluation on a test dataset my precision number is much higher so, I'm thinking this may be an issue with metrics calculation.

I'm using the following dataset:

To use this Dataset you need to:

        - Download coco dataset:
            annotations: http://images.cocodataset.org/annotations/annotations_trainval2017.zip
            train2017: http://images.cocodataset.org/zips/train2017.zip
            val2017: http://images.cocodataset.org/zips/val2017.zip

        - Unzip and organize it as below:
            coco
            ├── annotations
            │      ├─ instances_train2017.json
            │      ├─ instances_val2017.json
            │      └─ ...
            └── images
                ├── train2017
                │   ├─ 000000000001.jpg
                │   └─ ...
                └── val2017

Here is the command I am using:

python -m super_gradients.train_from_recipe \
        --config-name=coco2017_yolo_nas_s \
        dataset_params.train_dataset_params.data_dir=coco \
        dataset_params.train_dataloader_params.batch_size=45 \
        dataset_params.val_dataset_params.data_dir=coco \
        num_gpus=1 \
        multi_gpu=auto

Versions

super-gradients==3.7.1

[ShpihanVlad]

Most propably you are calculating these on score threshold 0.1, so from this comes low precision