Question on mAP and Raccoon Dataset

[sroj]

Hi! Thank you for this repo!

I'm trying to reproduce the training process on the raccoon dataset by following the steps on the readme file. My config file is very similar to the one provided there. The training ends after early stopping with very low loss values on the val dataset. However, the reported mAP is zero (see attached).

Is this expected? Is the mAP function supposed to be working? Asking because the TODO has an item relate to 'mAP evaluation'. So wondering if the mAP function is just not working.

[sroj]

Also, what was the configuration used to successfully train on the raccoon dataset? This is what I'm using (as per README.md):

{
    "model" : {
        "backend":         "Full Yolo",
        "input_size":           416,
        "anchors":              [0.57273, 0.677385, 1.87446, 2.06253, 3.33843, 5.47434, 7.88282, 3.52778, 9.77052, 9.16828],
        "max_box_per_image":    10,
        "labels":               ["raccoon"]
    },

    "train": {
        "train_image_folder":   "/u/srojas/mlcv_proj/raccoon_dataset/images/",
        "train_annot_folder":   "/u/srojas/mlcv_proj/raccoon_dataset/annotations/",

        "train_times":          10,
        "pretrained_weights":   "",
        "batch_size":           16,
        "learning_rate":        1e-4,
        "nb_epochs":             50,
        "warmup_epochs":        3,

        "object_scale":         5.0 ,
        "no_object_scale":      1.0,
        "coord_scale":          1.0,
        "class_scale":          1.0,
        "saved_weights_name":   "trained_weights_full.h5",

        "debug":                true
    },

    "valid": {
        "valid_image_folder":   "",
        "valid_annot_folder":   "",

        "valid_times":          1
    }
}

[experiencor]

Sorry I have no idea but the config looks right to me.

[sroj]

Thanks for replying! So the mAP evaluation function is indeed supposed to be working in the current version of the code?

[experiencor]

Yes, it's working.

[pure-water]

The issue has being raised here. https://github.com/experiencor/keras-yolo2/issues/246 It will be fixed after the PR merged

Regards

[sroj]

Hi @pure-water. Will test with your PR to double-check it fix the issue on my end as well. Thank you!

[pure-water]

It is already merged. Can you confirm it is fixing your issue as well? @sroj

[lychenpan]

@sroj I have the similar problem with you. Have you solved the problem? I have tested on the kangaroo datasets and raccoon datasets, both the same situation: mAP will decrease to zero after about 10 epochs. One thing is that i have change the 22 conv layers to untrainable.