"area" in annotations

[nico00]

Hi, I'm creating my own dataset but writing the annotations I've found a field called "area" that I didn't understand. According to my analysis, it doesn't refer to:

• image area (width x height)
• bounding box area (width x height)
• segmentation area

Any idea about how I should calculate it ? Thanks Nico

[tylin]

It means segmentation area. You can use mask.area function to compute it. https://github.com/pdollar/coco/blob/master/PythonAPI/pycocotools/mask.py#L35

[nico00]

Thanks tylin. As I have a polygon that describes the segmentation, to get the area value I have to:

• convert the polygon to a mask, with MaskApi.frPoly (I'm using LUA)
• encode the mask to RLE, with MaskApi.encode
• get the area, passing the RLE to MaskApi.area

Am I right ? Thanks Nico

[tylin]

That sounds right to me.

[nico00]

Hi tylin, sorry to bother you but I've not figured out how to solve this problem.

Basically I've to create an annotations JSON, structured in this way: { "annotations": [ { "area": ???, "bbox": [ 199.84, 200.46, 77.71, 70.88], "category_id": 1, "id": 156, "image_id": 558840, "iscrowd": 0, "segmentation": [ [ 239.97, 260.24, ..., 228.87, 271.34 ] ] }, ...

The only value that I miss is area. As you suggested I'm trying to calculate it using MaskApi.area This is what I wrote, just to give a try:

local coco = require 'coco'
maskApi = coco.MaskApi
polS = { torch.Tensor( { 239.97,  260.24,  222.04,  270.49,  199.84,  253.41,  213.5,  227.79,  259.62,  200.46,  274.13,  202.17,  277.55,  210.71,  249.37,  253.41,  237.41,  264.51,  242.54,  261.95,  228.87,  271.34  } ) }

local h = 427
local w = 640
local R = maskApi.frPoly(polS, h, w)
local Rs = maskApi.encode(R)  -- returns an error: attempt to call method 'size' 
local area = maskApi.area(Rs)
print (area)

maskApi.encode(R) returns "attempt to call method 'size'". Looking at the code of that function is seems that R has to be a 3D tensor but in my case is 2D.

I'm becoming crazy. Any suggestion? Thanks Nico

[tylin]

One segment can contain multiple polygons {poly1, poly2, ..., polyn}, I think that's the reason to cause error. You can load COCO annotations and print the polygon format in Torch to gain more understanding.

[nico00]

It seems that the problem was the sequence of calls. The right sequence should be:

local R = maskApi.frPoly(polS, h, w)
local area = maskApi.area(R)

The only detail I've noticed is that the area value I got, using LUA, is slightly different than the value I expected, according to some annotations you created (instance_val2014.json), for instance:

expected result: 2765.1486500000005
actual result: 2760

expected result: 1545.4213000000007
actual result: 1542

Does this mean my approach is wrong ? Or is this a consequence of using LUA instead of Python ? More important, will this cause problems during the training session ? Thanks Nico

[sifengyang]

@tylin Have you @nico00 solved the problem of 'area'? I experiment with your method. my code is this frPyObjects = _mask.frPyObjects segmentation = [[51.69, 292.27, 49.77, 241.33, 58.18, 240.13, 58.42, 292.03, 52.65, 292.51]] rs = frPyObjects( segmentation, 480, 640 ) output [{'counts': 'a_o0d0\\c0l0TO4LO1000O100``W8', 'size': [640L, 480L]}] How can I solve this mistake?

[AbuBakrCh]

@sifengyang This is not a mistake. After this you have to do: area = _mask.area a = area(rs)

[hazirbas]

those who need to write their own .json files can check my implementation for Davis 2016 benchmark:

coco-json-converter

[IaroslavS]

I hate RLE format. Someone wanted to make data lighter with the cost of lots of problems to other people.

[Johnqczhang]

It means segmentation area. You can use mask.area function to compute it. https://github.com/pdollar/coco/blob/master/PythonAPI/pycocotools/mask.py#L35

Hi @tylin, it seems that the "area" field of the prediction is given by the box area for the evaluation of object detection and keypoints detection task, according to line-338 and line-357 in the code snippet below: https://github.com/cocodataset/cocoapi/blob/8c9bcc3cf640524c4c20a9c40e89cb6a2f2fa0e9/PythonAPI/pycocotools/coco.py#L331-L359

When evaluating across different scales ("small", "medium", and "large"), both ground-truth (g) and prediction (d) will be either ignored or considered according to whether their "area" field is within the specific area range, https://github.com/cocodataset/cocoapi/blob/8c9bcc3cf640524c4c20a9c40e89cb6a2f2fa0e9/PythonAPI/pycocotools/cocoeval.py#L251-L252 https://github.com/cocodataset/cocoapi/blob/8c9bcc3cf640524c4c20a9c40e89cb6a2f2fa0e9/PythonAPI/pycocotools/cocoeval.py#L297-L298

So, given that the "area" field of ground-truth refers to the number of pixels in the segmentation mask, does this different processing between a ground-truth and a prediction correctly characterize the performance of an object or keypoints detector? As far as I know, there are many object instances of some categories (e.g., "person") of which the value of bbox area and segmentation area are quite different.

[BishwaBS]

@Johnqczhang Do we have updated repo with the updated code that considers the segmentation mask area instead of bbox area while evaluating "segm"?

[kadattack]

expected result: 2765.1486500000005 actual result: 2760

expected result: 1545.4213000000007 actual result: 1542

Does this mean my approach is wrong ? Or is this a consequence of using LUA instead of Python ?
More important, will this cause problems during the training session ?
Thanks
Nico

Has anyone figured out why this is happening with @nico00 results ? I noticed the same issue with instances_val2017.json And the same issue is for the BBox when using pycocotools._mask.toBbox() . The numbers seem to be off slightly. Will this cause problems?