Can't upload "COCO Keypoints 1.0" annotation to dataset

medphisiker commented 5 months ago

Actions before raising this issue

[X] I searched the existing issues and did not find anything similar.
[X] I read/searched the docs

Steps to Reproduce

I want to create classic dataset for keypoints with keypoints inside bbox. As mentioned before in this issue(link) I annotate a bounding box, annotate skeleton with keypoints and group it together using grouping feature.

Steps to reproduce:

Create a dataset with one picture. I use picture with fish.
Set skeleton for fish like this
Set label for rectangle ike this
Then we create this task. We annotate skeleton for 1 fish. We annotate bbox for the fish. After that we combine bbox with skeleton together using grouping tool.
Then we export our annotations to CVAT for images annotation format.
After that we export our annotation to "COCO Keypoints 1.0" annotation format.
We delete all annotations from our dataset in CVAT. Then we try to upload annotations to our dataset in CVAT from "CVAT for images 1.1" annotations file. Everything works correctly. We get the same labeling as we had before with one bbox and one skeleton for the fish which are groped together.
We delete all annotations from our dataset in CVAT. Then we try to upload annotations to our dataset in CVAT from "COCO Keypoints 1.0" annotations file. And we get this error message

Expected Behavior

I was hoping that the markup from the "COCO keypoint" format would also be readable.

Possible Solution

Unfortunately, I was unable to determine the cause of this error.

Context

I need to label up a dataset with skeleton's key points inside a bounding box in CVAT. Many models are trained using dataset's markup in the "MS COCO keypoint" format. Therefore, I need the ability to export and import this dataset correctly in this format.

Environment

git log -1

commit d427822868de5d2d223b7e7d043a08d42dda2212 (HEAD, tag: v2.10.2, origin/master)
Merge: a33f7f570 3733a071b
Author: cvat-bot[bot] <147643061+cvat-bot[bot]@users.noreply.github.com>
Date:   Fri Jan 26 18:14:54 2024 +0000

    Merge pull request #7401 from opencv/release-2.10.2

    Release v2.10.2

docker version


Client: Docker Engine - Community
Version:           24.0.7
API version:       1.43
Go version:        go1.20.10
Git commit:        afdd53b
Built:             Thu Oct 26 09:08:01 2023
OS/Arch:           linux/amd64
Context:           default

Server: Docker Engine - Community Engine: Version: 24.0.7 API version: 1.43 (minimum version 1.12) Go version: go1.20.10 Git commit: 311b9ff Built: Thu Oct 26 09:08:01 2023 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.6.24 GitCommit: 61f9fd88f79f081d64d6fa3bb1a0dc71ec870523 runc: Version: 1.1.9 GitCommit: v1.1.9-0-gccaecfc docker-init: Version: 0.19.0 GitCommit: de40ad0

- sb_release -a

No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04.6 LTS Release: 20.04 Codename: focal

zhiltsov-max commented 4 months ago

Hi,

I was able to reproduce the problem. The problem is that CVAT exports skeletons with boxes annotated, but the resulting label used is from the bbox instead of the skeleton. You can change the label id in the annotations to the skeleton's one, but then CVAT will not be able to import the bbox (as it needs to have a different class in CVAT). This should be fixed, but I don't see a quick workaround for this now.

medphisiker commented 4 months ago

Hello, thank you for your reply. I want to create classic dataset for keypoints with keypoints inside bbox. As mentioned before in this issue(https://github.com/opencv/cvat/issues/7411) I annotate a bounding box, annotate skeleton with keypoints and group it together using grouping feature.

I annotate the bbox and group it with the skeleton to set the coordinates of the bbox to the bbox of the skeleton. When I export this combo from skeleton and bbox I get the right MS COCO keypoint annotation. There are two categories in MS COCO keypoint annotation file:

bbox "treska_bbox"
skeleton "treska"

    "categories": [
        {
            "id": 1,
            "name": "treska_bbox",
            "supercategory": "",
            "keypoints": [],
            "skeleton": []
        },
        {
            "id": 2,
            "name": "treska",
            "supercategory": "",
            "keypoints": [
                "eye",
                "cutt",
                "belly"
            ],
            "skeleton": [
                [
                    2,
                    3
                ],
                [
                    1,
                    2
                ]
            ]
        }
    ],

But annotations have only one of them:

skeleton "treska"

"annotations": [
    {
        "id": 1,
        "image_id": 1,
        "category_id": 1,
        "segmentation": [],
        "area": 102402.0364,
        "bbox": [
            567.9,
            589.0,
            521.98,
            196.18
        ],
        "iscrowd": 0,
        "attributes": {
            "occluded": false,
            "rotation": 0.0
        },
        "keypoints": [
            614.93,
            702.46,
            2,
            727.64,
            710.61,
            2,
            777.84,
            680.77,
            2
        ],
        "num_keypoints": 3
    }
]
}

Annotation's field "bbox" have coordinates from bbox of "treska_bbox", because I combined them in group as mentioned in this issue(https://github.com/opencv/cvat/issues/7411).

I don't need the bboxes by themselves. I can remove the bbox "treska_bbox" from the categories in the annotation file. But the problem is that when I reading MS COCO key points annotation, as I understand it, CVAT convert skeletons bbox to skeleton's zoom/transform bbox. And I get bbox of skeleton that feets keypoints in skeleton )

306543688-56461a1a-b1c5-4bce-8380-2256c942f111

medphisiker commented 4 months ago

I tested the method you suggested

You can change the label id in the annotations to the skeleton's one, but then CVAT will not be able to import the bbox (as it needs to have a different class in CVAT)

As a result, only skeleton remains, as you said. Annotation's field "bbox" have coordinates from bbox of "treska_bbox", because I combined them in group as mentioned in this issue(https://github.com/opencv/cvat/issues/7411).

I guess they've changed because of

 I reading MS COCO key points annotation, as I understand it, CVAT convert skeletons bbox to skeleton's zoom/transform bbox. And I get bbox of skeleton that feets keypoints in skeleton )

Here is the result of the import Screenshot_2

zhiltsov-max commented 2 weeks ago

cvat-ai / cvat