Panoptic Segmentation for help !

[ranjiewwen]

thanks for you share; recently i am also interest in Panoptic Segmentation; now i also train Mask R-CNN and Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation model for cityscapes; and i also run this deeplab-pytorch model; but I have some doubts about this task ;

• fisrt how you get the instance ans semantic label from panoptic_annotations_trainval2017 annotations, it only give the Panoptic Segmentation label ? i find the panoptic-api ,it has panoptic2detection_coco_format.py and panoptic2semantic_segmentation.py scripts, Do you convert in this way ？or can you share you datasets processing method ?

• second i see the toolbox code is the original code , can you share the new ? it can apply on coco-2017-panoptic segmentation task .

• third i see the panoptic-api combine label which should have coco format json file for semseg_json_file and instseg_json_file; how you process the problem ?

• thanks again, looking for you reply!

[m3at]

Glad you're interested in tackling panoptic segmentation too! First an apology, my code is unpolished and not self explanatory, improving it is in my backlog and will be tackled as time allows.

For your questions:

get the instance and semantic label

For a quick visual inspection, the easiest is to load instance/semantic only datasets and only look at what interests you. To get the exact instance labels is not possible by design for the panoptic task (resolve overlaps), but you can get an approximation by looking at classes ID and separate accordingly (see 2. Classes).

toolbox code is the original code , can you share the new ?

I'm not sure I get your question sorry. Current code is for panoptic task, code in toolbox is borrowed from older coco challenges but adapted for panoptic and should work as is. If not, please raise a specific issue and I'll do my best to fix it :)

combine label which should have coco format json file

I follow section "7. Machine Performance Baselines" of the paper. You can see an example of combining predictions and output in the required evaluation format in this notebook.