calculate MaxBoxAcc

[shadowwkl]

Hi,

I try to calculate the MaxBoxAcc version 1 for CAM method on CUB dataset, using VGG16, the number (~85%) is much higher than the number reported in the paper (~76%).

I calculate it as:

counter = 0 for all testing_image:         get the current CAM and normalized it via min-max normalization (range to [0, 1])         for 1,000 steps (I assume you sample the score map threshold 1,000 steps):                 c_CAM = c_CAM >= current_score_map_threshold                 get the current bbox and calculate the IOU         if one of IOUs > 0.5 (there shall be 1,000 IOUs):                counter += 1

and the final maxboxacc = counter / number_of_testing_images.

Do I miss something in the procedure? Btw, the bbox is estimated from all contours.

[junsukchoe]

Thanks for your interest in our work!

For MaxBoxAccV1, the bbox is extracted from the largest connected contour in score map. And here is our evaluation algorithm:

counter = np.zeros(len(threshold_list))
for all testing_image:
    get the current CAM and normalized it via min-max normalization (range to [0, 1])
    for idx, threshold in enumerate(range(len(threshold_list))):
        c_CAM = c_CAM >= threshold
        get the current bbox and calculate the IOU
        if IoU > 0.5:
            counter[idx]+=1
localization_accuracies = (counter * 100 / number_of_testing_images)
MaxBoxAccV1 = localization_accuracies.max()

For more detail, please see this: https://github.com/clovaai/wsolevaluation/blob/e00842f8e9d86588d45f8e3e30c237abb364bba4/evaluation.py#L217

[shadowwkl]

Thanks for your interest in our work!

For MaxBoxAccV1, the bbox is extracted from the largest connected contour in score map. And here is our evaluation algorithm:

counter = np.zeros(len(threshold_list))
for all testing_image:
    get the current CAM and normalized it via min-max normalization (range to [0, 1])
    for idx, threshold in enumerate(range(len(threshold_list))):
        c_CAM = c_CAM >= threshold
        get the current bbox and calculate the IOU
        if IoU > 0.5:
            counter[idx]+=1
localization_accuracies = (counter * 100 / number_of_testing_images)
MaxBoxAccV1 = localization_accuracies.max()

For more detail, please see this: https://github.com/clovaai/wsolevaluation/blob/e00842f8e9d86588d45f8e3e30c237abb364bba4/evaluation.py#L217

Thanks for your quick reply! I got it.