wondering the meaning of id in segments_info

[baidingyuan]

hello jiakang @Jingkang50 , firstly thanks for the wonderful work, i have some questions when trying this program, hope to have some answers from you. 1: what does this id in segments_info mean? for example: {'area': 42555, 'category_id': 0, 'id': 3648962, 'iscrowd': 0, 'isthing': 1}, i try to compare it with instances_train2017, only found they mismatch, did you generate ids by yourself? 2:if i want to train psgtr on my own custom dataset ,what data should i prepare hope to receive your answer, thanks in advance

[Jingkang50]

Thank you for your questions.

1. I did not generate ids by myself, they are directly from coco. seg_ids can be connected with masks in the mask png files. You can refer to https://github.com/Jingkang50/OpenPSG/blob/main/tools/Visualize_Dataset.ipynb
2. you can also refer to our huggingface demo or https://github.com/Jingkang50/OpenPSG/blob/main/predict.py for your custom data.

[baidingyuan]

Thank you for your questions.

1. I did not generate ids by myself, they are directly from coco. seg_ids can be connected with masks in the mask png files. You can refer to https://github.com/Jingkang50/OpenPSG/blob/main/tools/Visualize_Dataset.ipynb
2. you can also refer to our huggingface demo or https://github.com/Jingkang50/OpenPSG/blob/main/predict.py for your custom data.

well... I'm still confused about 1, for example: this data in psg_train_val.json: {"file_name": "train2017/000000417720.jpg", "height": 640, "width": 480, "image_id": "107902", "pan_seg_file_name": "panoptic_train2017/000000417720.png", "segments_info": [{"id": 3648962, "category_id": 0, "iscrowd": 0, "isthing": 1, "area": 42555}, {"id": 3293004, "category_id": 0, "iscrowd": 0, "isthing": 1, "area": 45880}, {"id": 4470325, "category_id": 25, "iscrowd": 0, "isthing": 1, "area": 22774}, {"id": 4545879, "category_id": 116, "iscrowd": 0, "isthing": 0, "area": 48824}, {"id": 7633023, "category_id": 117, "iscrowd": 0, "isthing": 0, "area": 44811}, {"id": 14079701, "category_id": 119, "iscrowd": 0, "isthing": 0, "area": 3417}, {"id": 3504233, "category_id": 125, "iscrowd": 0, "isthing": 0, "area": 97238}], "relations": [[0, 1, 2], [0, 6, 14], [1, 2, 21], [1, 6, 14], [4, 3, 1], [5, 3, 0]], "annotations": [{"bbox": [49.0, 162.0, 207.0, 617.0], "bbox_mode": 0, "category_id": 0}, {"bbox": [259.0, 158.0, 397.0, 631.0], "bbox_mode": 0, "category_id": 0}, {"bbox": [157.0, 75.0, 469.0, 279.0], "bbox_mode": 0, "category_id": 25}, {"bbox": [0.0, 0.0, 480.0, 131.0], "bbox_mode": 0, "category_id": 116}, {"bbox": [0.0, 112.0, 480.0, 323.0], "bbox_mode": 0, "category_id": 117}, {"bbox": [215.0, 0.0, 480.0, 54.0], "bbox_mode": 0, "category_id": 119}, {"bbox": [0.0, 285.0, 480.0, 640.0], "bbox_mode": 0, "category_id": 125}], "coco_image_id": "417720"} it seems that there're 7 objects in this image, and their segments_info ids are 3648962/3293004/4470325/4545879/7633023/147079701/3504233 and this data is related to these ones in coco/instances_train2017.json: {"segmentation": [[156.06,98.93,206.17,83.18,214.77,80.32,244.83,77.45,276.33,74.59,309.26,86.04,335.03,97.5,356.51,100.36,370.83,116.11,400.89,131.86,423.8,140.45,433.83,146.18,451.01,177.68,469.62,207.74,462.46,209.18,456.73,236.38,412.35,219.2,379.42,220.63,366.53,214.9,352.21,186.27,342.19,166.22,307.83,157.63,289.22,163.36,272.04,177.68,236.24,156.2,201.88,141.88,180.4,127.57,164.65,116.11,160.36,103.23],[297.81,243.54,277.76,279.33,274.9,277.9,289.22,237.81,293.51,239.24]],"area": 22777.60444999998,"iscrowd": 0,"image_id": 417720,"bbox": [156.06,74.59,313.56,204.74],"category_id": 28,"id": 281762}

{"segmentation": [[271.82,185.62,277.57,153.98,319.28,153.98,332.22,153.98,345.17,172.68,352.36,194.25,365.3,241.71,369.62,261.85,379.69,286.3,386.88,338.07,396.94,375.47,389.75,414.3,385.44,441.62,396.94,487.65,395.51,509.22,381.12,563.87,368.18,619.96,291.96,631.47,276.13,568.19,286.2,507.78,290.52,467.51,291.96,450.25,273.26,443.06,264.63,388.41,279.01,361.08,289.08,348.14,293.39,330.88,289.08,326.57,258.88,303.56,264.63,274.79,287.64,276.23,299.15,241.71]],"area": 46413.684150000016,"iscrowd": 0,"image_id": 417720,"bbox": [258.88,153.98,138.06,477.49],"category_id": 1,"id": 486278}

{"segmentation": [[103.32,579.73,119.1,542.42,91.84,462.06,76.05,404.66,84.66,353.0,73.18,295.61,74.62,243.95,70.31,220.99,48.79,206.64,55.96,180.81,91.84,163.59,123.41,162.15,150.67,195.16,154.98,226.73,189.42,243.95,186.55,380.27,206.64,528.07,205.2,614.17,182.24,617.04,177.94,584.04,156.41,529.51,136.32,582.6,103.32,589.78]],"area": 42575.6284,"iscrowd": 0,"image_id": 417720,"bbox": [48.79,162.15,157.85,454.89],"category_id": 1,"id": 486514} I got problems here: there are only 3 objects in the coco annotations ,perhaps you have annotated 4 new objects, but for these 3 objects, their ids are 281762/486278/486514, they mismatch the ids in psg_train_val.json, looking forward to your answer @Jingkang50

[Jingkang50]

Actually we are using files for coco panoptic segmentation rather than instance segmentation. So I guess the 4 more things you find is stuff.

[baidingyuan]

yes, that's it, thanks again

Actually we are using files for coco panoptic segmentation rather than instance segmentation. So I guess the 4 more things you find is stuff.

[baidingyuan]

i have examined the code that generates segments IDs, still have a problem: it seems that the code generate id depend instances's color , i use coco annotator to annotate my data, if two instances belong to the same category, their colors are same, then their IDs are same , so are the instances with same category but in different pics. just wonder how to solve this problem. I found that the training process requires the use of psg.json, original images, and their corresponding png files at the same time. Wouldn't it be more efficient to only use json and images? @Jingkang50

and the rgbtoid code is : def rgb2id(color): if isinstance(color, np.ndarray) and len(color.shape) == 3: if color.dtype == np.uint8: color = color.astype(np.int32) return color[:, :, 0] + 256 color[:, :, 1] + 256 256 color[:, :, 2] return int(color[0] + 256 color[1] + 256 256 color[2])