Support of (Key-)Point formats in CVAT

[JParzival]

Hello everyone,

the issue itself is that, when I export annotations in COCO Format, points are not shown in the json, but polygons are. I have tried both creating a new task and only annotating points and also combining points and polygons, and neither of them exports the COCO Format with both instances.

Expected Behaviour

The expected behaviour is to be able to export the COCO Format with both points and polygons.

Current Behaviour

The current behaviour is that the COCO Format is not able to export points.

Possible Solution

It would just be a matter of implementing the points inside the COCO Format, so that they can be exported alongside the polygons.

Steps to Reproduce (for bugs)

1. Create a new task
2. Set a point
3. Export in COCO Format

Context

The project that I am actually working in requires both a combination of a mask and a keypoint inside of it, in the most important part of the object. Without the points, the goal cannot be accomplished.

Your Environment

• Docker version docker version (e.g. Docker 17.0.05): Docker version 19.03.12, build 48a66213fe
• Are you using Docker Swarm or Kubernetes? No, just Docker (required for the application)
• Operating System and version (e.g. Linux, Windows, MacOS): Windows
• Code example or link to GitHub repo or gist to reproduce problem: https://github.com/openvinotoolkit/cvat

[claverru]

I'm also interested in this.

[taranais]

I am looking for this

[zhiltsov-max]

COCO keypoints are not supported yet, possible solutions are described here: https://github.com/openvinotoolkit/cvat/issues/1784 and https://github.com/openvinotoolkit/cvat/issues/1421#issuecomment-616576420

[claverru]

COCO keypoints are not supported yet, possible solutions are described here: openvinotoolkit/cvat#1784 and #1421 (comment)

@zhiltsov-max I think those workarounds don't work for me because I have less than 3 keypoints, and masks are only supported with at least 3 points.

[claverru]

@zhiltsov-max Also, I think having keypoints in COCO format should be a very common feature.

[JParzival]

Yes, I agree with @claverru, having the keypoints in COCO is a very common feature, and in my opinion is an urgent enhancement

[zhiltsov-max]

https://github.com/openvinotoolkit/cvat/issues/1421#issuecomment-616576420 describes exactly this topic.

[ValerianGonnot]

I'm also interested in this.

[zhiltsov-max]

Could you describe the expected output for the points from CVAT when exporting in COCO format? If you want to export them as is, it is possible already, using the way described above.

[JParzival]

Hi @zhiltsov-max , this is the exact format that is needed:

{ "info": { "contributor": "", "date_created": "", "description": "", "url": "", "version": "", "year": "" }, "images": [ { "id": 1, "width": 1920, "height": 1080, "file_name": "image1.tiff", "license": 0, "flickr_url": "", "coco_url": "", "date_captured": 0 }, ], "licenses": [ { "name": "", "id": 0, "url": "" } ], "categories": [ { "id": 1, "name": "cat1", "supercategory": "" }, { "id": 2, "name": "cat2", "supercategory": "" } ], "annotations": [ { "id": 1, "image_id": 1, "category_id": 1, "segmentation": [ [ 1677.25, 57.33, 1677.55, 55.03, 1679.15, 52.73, 1680.75, 50.53, 1677.25, 57.33 ] ], "area": 277.0, "bbox": [ 1677.25, 35.93, 23.5, 27.2 ], "iscrowd": 0, "attributes": { "occluded": false }, "num_keypoints": 1, "keypoints": [ 1689.55, 50.33, 2 ] } ] }

Thank you very much

[nmanovic]

@kirill-sizov , it is a very hot issue.

[natelee-tw]

Can consider trying this:

• Exporting CVAT dataset as Datumaro 1.0 format, including save images
• Unzip the dataset
• Create a new folder, datum create, followed by datum import [path of unzipped dataset]
• datum export -f coco_person_keypoints source-1

[yasakova-anastasia]

Keypoints have been added to CVAT and COCO formats.

[oz-swapnil]

This issue arose again in the latest version of CVAT.

[nmanovic]

@yasakova-anastasia , could you please check?

[yasakova-anastasia]

@oz-swapnil, could you please describe your problem in more detail? Keypoints are supported in CVAT and COCO Keypoints formats.

[oz-swapnil]

@yasakova-anastasia Sure. I have annotated points with multiple labels in 56 images, saved the task and tried to export the dataset as COCO Keypoints 1.0. In the JSON file, the annotations field was empty. "annotations":[]

[yasakova-anastasia]

This format can only export skeleton annotations. Do you have skeleton annotations? If you want to know more about skeletons, you can watch this video.

[oz-swapnil]

Okay. Got it. Thanks for the help! But can't we get individual points in COCO format?

[zhiltsov-max]

COCO requires categories for points, and plain points do not have any. This is the reason why skeletons are required.

[oz-swapnil]

Got it @zhiltsov-max! Thank you for the clarification and thank you @yasakova-anastasia and @nmanovic for helping out. This issue can be closed now.