Objective Detection using COCO classes: first label placed to the end of the list

jinzishuai commented 4 years ago

My actions before raising this issue

[X] Read/searched the docs
[X] Searched past issues

Expected Behaviour

I am training a model using YOLOv5 and the objects are labelled using the 80 classes of COCO:

names: ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light',
        'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow',
        'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee',
        'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard',
        'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple',
        'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch',
        'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone',
        'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear',
        'hair drier', 'toothbrush']

I am having a bit of challengens creating a labeling task in CVAT. I am able to generate the 80 labels in JSON format but it seems that CVAT will reorder them labels by placing the first label in the last, therefore breaking the original order of COCO.

Current Behaviour

the labels provided to CVAT are in the original COCO order

CVAT re-order them like


[
{
"name": "bicycle",
"id": 3,
"color": "#f13fb8",
"attributes": []
},
{
"name": "car",
"id": 4,
"color": "#60f58f",
"attributes": []
},
{
"name": "motorcycle",
"id": 5,
"color": "#ec669a",
"attributes": []
},

... ]


![image](https://user-images.githubusercontent.com/1074685/91315992-911ce000-e775-11ea-8a48-79d8ed1077f5.png)

Note that "person" is the first entry in the COCO list.

### Possible Solution
<!--- Not obligatory, but suggest a fix/reason for the bug, or ideas how
to implement the addition or change -->

It seems that we have a bug in the placement of the first label to the very end.

### Steps to Reproduce (for bugs)
<!--- Provide a link to a live example, or an unambiguous set of steps to
reproduce this bug. Include code to reproduce, if relevant -->
1. create a new task in CVAT with 3 labels in the order of "person", "bicycle", "car" or just upload the following json
```json
[
  {
    "name": "person",
    "color": "#330df0",
    "attributes": []
  },
  {
    "name": "bicycle",
    "color": "#4226d1",
    "attributes": []
  },
  {
    "name": "car",
    "color": "#ff6037",
    "attributes": []
  }
]

observe that the resulting task reordered the labels: first label is moved to the last

[
{
"name": "car",
"id": 45449,
"color": "#ff6037",
"attributes": []
},
{
"name": "bicycle",
"id": 45448,
"color": "#4226d1",
"attributes": []
},
{
"name": "person",
"id": 45447,
"color": "#330df0",
"attributes": []
}
]

Context

It is my understanding that the order of the labels are important for machine learning trainings since in the end, each label is represented as a integer number such as 41 for the "cup" class. If the data is now labelled with a different number, then the model will be completely wrong.

Your Environment

https://cvat.org/tasks/10146

Next steps

You may join our Gitter channel for community support.

jinzishuai commented 4 years ago

Please note that I originally thought CVAT reorders the labels alphabetically but after further look at the resulting labels (screenshot above), they are actually not alphabetic either.

In fact, the problem is only that the first entry is moved to the very last while the order of the other labels remain unchanged.

ActiveChooN commented 2 years ago

@jinzishuai, can you please check if #3987 would resolve the issue?

cvat-ai / cvat