mAP calculation

[abhigoku10]

@dbolya in the eval.py have you included the mAP calculation on the validation set / test set ?? if you is there any cmmd to be used to calculate the mAP for custom dataset

[dbolya]

See the Quantitative Results on COCO section in the ReadMe for instructions on both validation and test set mAP calculations. It will use whatever dataset you have set in the config, so it will work for custom datasets too.

[abhigoku10]

@dbolya thanks for the pointers

[abhigoku10]

@dbolya can i get mAP of individual classes from your source code , if so how to get it

[dbolya]

Here is where I calculate the mAP for each triple of class, iou type, and iou threshold: https://github.com/dbolya/yolact/blob/6e22d1b1ccbeab4001bc6acd9124f5a679083d21/eval.py#L933

If you reorder the data and how it's stored, you can preserve the class part. You just have to average per class instead of average all the classes together like I do there.

[timdah]

Just want to put my quick solution in here, in case somebody else wants mAP per class also.

Replace the calc_map method in eval.py with this one:

def calc_map(ap_data):
    print('Calculating mAP...')
    aps = [{'box': [], 'mask': []} for _ in iou_thresholds]

    for _class in range(len(cfg.dataset.class_names)):
        for iou_idx in range(len(iou_thresholds)):
            for iou_type in ('box', 'mask'):
                ap_obj = ap_data[iou_type][iou_idx][_class]

                if not ap_obj.is_empty():
                    aps[iou_idx][iou_type].append(ap_obj.get_ap())
    ################
    ### addition ###
    ################
        all_maps = {'box': OrderedDict(), 'mask': OrderedDict()}
        for iou_type in ('box', 'mask'):
            all_maps[iou_type]['all'] = 0  # Make this first in the ordereddict
            for i, threshold in enumerate(iou_thresholds):
                mAP = aps[i][iou_type][_class] * 100 if len(aps[i][iou_type]) > 0 else 0
                all_maps[iou_type][int(threshold * 100)] = mAP
            all_maps[iou_type]['all'] = (sum(all_maps[iou_type].values()) / (len(all_maps[iou_type].values()) - 1))
        print('#################### Class:', cfg.dataset.class_names[_class], '####################')
        print_maps(all_maps)
    ####################
    ### addition end ###
    ####################

    all_maps = {'box': OrderedDict(), 'mask': OrderedDict()}

    # Looking back at it, this code is really hard to read :/
    for iou_type in ('box', 'mask'):
        all_maps[iou_type]['all'] = 0 # Make this first in the ordereddict
        for i, threshold in enumerate(iou_thresholds):
            mAP = sum(aps[i][iou_type]) / len(aps[i][iou_type]) * 100 if len(aps[i][iou_type]) > 0 else 0
            all_maps[iou_type][int(threshold*100)] = mAP
        all_maps[iou_type]['all'] = (sum(all_maps[iou_type].values()) / (len(all_maps[iou_type].values())-1))

    print('#################### All Classes ####################')  # also added
    print_maps(all_maps)
    return all_maps

[denashamss]

Hello @timdah do you know how can I calculate precision and recall for each class? thank you

[denashamss]

Hello all, do you know how can I get precision and recall for each class? thank you