how to calculate score in result-dataset?

[l1ndaB]

I try to evaluate a trained keypoint detection model with the coco-evaluation in python. The result-dataset need to have a "score" of type float (see http://cocodataset.org/#format-results - Point "2. Keypoint Detection": [{ "image_id": int, "category_id": int, "keypoints": [x1,y1,v1,...,xk,yk,vk], "score": float, }] Note: keypoint coordinates are floats measured from the top left image corner (and are 0-indexed). We recommend rounding coordinates to the nearest pixel to reduce file size. Note also that the visibility flags vi are not currently used (except for controlling visualization), we recommend simply setting vi=1.).

The only thing I can find about this score is, that it should be the confidence of the detected keypoints. But I have 17 keypoints and just one score. Does anybody know how this score is calculated? Is it the average of the confidences of all keypoints? And do I have to normalize the score to [0,1] or can it be between [-inf, inf]?

Thank you in advance!

[rebeen]

Hi, did you find how to get the score ?? I am doing object detection and with a pre-trained model but don't know how to get the following format

[{
"image_id": int, "category_id": int, "bbox": [x,y,width,height], "score": float,
}]

[l1ndaB]

Hi, no I didn't find out how the score needs to be calculated. So I just used the average of the confidences of all keypoints for my project. But I still don't know if that is correct...

[yevhen-k]

Scores for keypoints, as far as I know, are the same as for bounding boxes. See https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/cocoeval.py#L197 and https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/cocoeval.py#L173

[JonnoNield]

Scores for keypoints, as far as I know, are the same as for bounding boxes. See https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/cocoeval.py#L197 and https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/cocoeval.py#L173

this does not answer how score is calculated.

[sdu-gyf]

hi, score is the average of the confidences of all keypoints multiplied by person detected box's score hope this can help you

[sdu-gyf]

for example:

{"keypoints": [
            307.539306640625,
            204.469482421875,
            0.7098438739776611,
            309.46649169921875,
            205.43309020996094,
            0.7774471044540405,
            311.3937072753906,
            202.54229736328125,
            0.6868460178375244,
            306.5757141113281,
            211.21466064453125,
            0.6341773867607117,
            312.3572998046875,
            200.61509704589844,
            0.6822942495346069,
            305.61212158203125,
            221.81422424316406,
            0.43058279156684875,
            306.5757141113281,
            198.6879119873047,
            0.43003445863723755,
            287.3038024902344,
            211.21466064453125,
            0.3350619673728943,
            305.61212158203125,
            189.05194091796875,
            0.6003181338310242,
            291.1581726074219,
            191.94273376464844,
            0.4339810609817505,
            293.0853576660156,
            189.05194091796875,
            0.5613468885421753,
            285.3765869140625,
            220.8506317138672,
            0.5778497457504272,
            292.12176513671875,
            207.3602752685547,
            0.3434161841869354,
            276.7042236328125,
            200.61509704589844,
            0.8254815340042114,
            304.6485290527344,
            188.08834838867188,
            0.559004008769989,
            261.28668212890625,
            197.7243194580078,
            0.6696884036064148,
            285.3765869140625,
            182.30677795410156,
            0.4902821183204651
        ]}
{"bbox": [249.8199079291458, 175.21093805640606, 74.00419360691592, 55.626325589288854], "category_id": 1, "image_id": 532481, "score": 0.9992738366127014}

score is (.7098438739776611+.7774471044540405+.6868460178375244+.6341773867607117+.6822942495346069+.43058279156684875+.43003445863723755+.3350619673728943+.6003181338310242+.4339810609817505+.5613468885421753+.5778497457504272+.3434161841869354+.8254815340042114+.559004008769989+.6696884036064148+.4902821183204651)/17*0.9992738366127014=0.5729751492522307

[yijiudd]

if I want to calculate the score in keypoints results, does it mean that first I need to calculate the score of bbox, but where does the score of bbox come from?