Open LaoYang1994 opened 5 years ago
We generate gt_masks here: https://github.com/uber-research/UPSNet/blob/2ced987576a98dc16c5a580839b657948644d44d/upsnet/dataset/cityscapes.py#L179 and here: https://github.com/uber-research/UPSNet/blob/2ced987576a98dc16c5a580839b657948644d44d/upsnet/dataset/coco.py#L160 . It will only contain 0 and 1.
For your second question, a basic assumption is that there is no overlapping between each instance annotation. Both for official panoptic gt and our generated panoptic gt.
@YuwenXiong I test your code. There indeed exists the two problem. https://github.com/uber-research/UPSNet/blob/2ced987576a98dc16c5a580839b657948644d44d/upsnet/dataset/base_dataset.py#L941 https://github.com/uber-research/UPSNet/blob/2ced987576a98dc16c5a580839b657948644d44d/upsnet/dataset/base_dataset.py#L942 In this place, the small pictures will be padded 255.
For the second question, you can check the picture of id 69816.
Thanks for pointing it out, I confirm that these two issues exist. The first one will generate wrong panoptic gt when image shape cannot be divided by 32 (which does happen in coco) and will give slightly wrong supervision for panoptic loss. I'll fix it now. The second one is a little bit tricky, it comes from the bad instance annotation quality of coco. A potential solution is recalculating instance masks from official panoptic gt instead of get masks from original instance annotation.
https://github.com/uber-research/UPSNet/blob/2ced987576a98dc16c5a580839b657948644d44d/upsnet/operators/modules/mask_matching.py#L52 In this line, you just use "gt_masks[[i], :, :] != 0" to judge whether a pixel belongs to the instance. But the picture is padded using 255. I think there should be another rule: "& gt_masks[[i], :, :] != 255". Do you think what I pointed is right?
Another question is the overlap relation. There may be a big table in the picture. If the table is the last instance, then the whole panoptic gt will be covered by the correspond "id". This is not the panoptic gt we want, isn't it?
Looking forwark for your reply.