Incorrect format of the COCO json export

grin commented 4 years ago

I created some keypoint annotations which I want to export to the COCO json format. But it seems the export format for keypoints is incorrect. Currently the export I'm getting for my keypoints looks like this:

{"images":[
{"id":1,"file_name":"1.jpg","width":2047,"height":1537},
{"id":2,"file_name":"10.jpg","width":1500,"height":945},
{"id":3,"file_name":"2.jpg","width":3072,"height":2057},
{"id":4,"file_name":"3.jpg","width":1500,"height":1101},
{"id":5,"file_name":"4.jpg","width":1600,"height":1001},
{"id":6,"file_name":"5.jpg","width":1600,"height":1500},
{"id":7,"file_name":"6.jpg","width":1242,"height":2208},
{"id":8,"file_name":"7.jpg","width":1200,"height":1800},
{"id":9,"file_name":"8.jpg","width":2048,"height":1536},
{"id":10,"file_name":"9.jpg","width":2048,"height":1365}],
"annotations":[
{"area":0,"iscrowd":0,"id":1,"image_id":1,"category_id":10,"segmentation":[[1091,684]],"bbox":[1091,684,1,1]},
{"area":0,"iscrowd":0,"id":2,"image_id":1,"category_id":9,"segmentation":[[1187,361]],"bbox":[1187,361,1,1]},
{"area":0,"iscrowd":0,"id":3,"image_id":1,"category_id":8,"segmentation":[[1115,341]],"bbox":[1115,341,1,1]},
{"area":0,"iscrowd":0,"id":4,"image_id":1,"category_id":7,"segmentation":[[944,690]],"bbox":[944,690,1,1]},
{"area":0,"iscrowd":0,"id":5,"image_id":1,"category_id":6,"segmentation":[[861,366]],"bbox":[861,366,1,1]}
// ....
}

where each keypoint is represented as a bbox/segmentation, while it should be in this format:

  "images": [
    {"id":579902,"license":3,"file_name":"000000579902.jpg"},
    // ...
  ],
  "annotations": [
    {
      "segmentation": [[321.62,243.74, /*...*/]],
      "num_keypoints": 14,
      "area": 17702.97795,
      "iscrowd": 0,
      "keypoints": [215,485,2,206,479,2,223,451,2,335,492,2,346,489,2,328,455,2, /*...*/],
      "image_id": 579902,
      "bbox": [200.03,232.57,182.53,270.8],
      "category_id": 1,
      "id": 191310
    },
    {
      "segmentation": [204.01,306.23,/*...*/],
      "num_keypoints": 15,
      "area": 5463.6864,
      "iscrowd": 0,
      "keypoints": [244,408,2,250,407,2,256,403,2,214,432,2,214,428,2,226,433,2, /*...*/],
      "image_id": 289343,
      "bbox": [204.01,235.08,60.84,177.36],
      "category_id": 1,
      "id": 201376
    },

with keypoints as an array of integers: [x1, x2, v1, x2, y2, v2, ...] (v indicates visibility— v=0: not labeled (in which case x=y=0), v=1: labeled but not visible, and v=2: labeled and visible )

ryouchinsa commented 4 years ago

Thanks for writing the issue.

Currently each point is saved as a polygon which includes a point. You mean that you need to save a group of points as "keypoints".

Look at this COCO document. http://cocodataset.org/#format-data

For "Object Detection", "keypoints" is not included. For "Keypoint Detection", "keypoints" is included. So that RectLabel needs to update for the "Keypoint Detection".

Could you show us the detailed use-case of the "Keypoint Detection"? Please show us the screenshot of the image and points you labeled.

grin commented 4 years ago

Sure, I'll send some examples to support@rectlabel.com (tomorrow) as I would prefer to avoid posting them here publicly.

grin commented 4 years ago

...and yes, keypoint detection is my use case.

ryouchinsa commented 4 years ago

The new update version 3.01.5 was released. To show the new update on Mac App Store, press command + R to reload.

Added "Draw keypoints with a skeleton" on Edit menu.

Create keypoints https://rectlabel.com/help#create_keypoints

Export XML files to COCO JSON file https://rectlabel.com/help#xml_to_coco

keypoints

ryouchinsa commented 4 years ago

Currently If you need our support to fix this problem, please let us know.

ryouchinsa / Rectlabel-support

Incorrect format of the COCO json export #162