search

uber-research / UPSNet

UPSNet: A Unified Panoptic Segmentation Network
Other
648 stars 120 forks source link

A question about panoptic gt #32

Open LaoYang1994 opened 5 years ago

LaoYang1994 commented 5 years ago

https://github.com/uber-research/UPSNet/blob/2ced987576a98dc16c5a580839b657948644d44d/upsnet/operators/modules/mask_matching.py#L52 In this line, you just use "gt_masks[[i], :, :] != 0" to judge whether a pixel belongs to the instance. But the picture is padded using 255. I think there should be another rule: "& gt_masks[[i], :, :] != 255". Do you think what I pointed is right?

Another question is the overlap relation. There may be a big table in the picture. If the table is the last instance, then the whole panoptic gt will be covered by the correspond "id". This is not the panoptic gt we want, isn't it?

Looking forwark for your reply.

YuwenXiong commented 5 years ago

We generate gt_masks here: https://github.com/uber-research/UPSNet/blob/2ced987576a98dc16c5a580839b657948644d44d/upsnet/dataset/cityscapes.py#L179 and here: https://github.com/uber-research/UPSNet/blob/2ced987576a98dc16c5a580839b657948644d44d/upsnet/dataset/coco.py#L160 . It will only contain 0 and 1.

For your second question, a basic assumption is that there is no overlapping between each instance annotation. Both for official panoptic gt and our generated panoptic gt.

LaoYang1994 commented 5 years ago

@YuwenXiong I test your code. There indeed exists the two problem. https://github.com/uber-research/UPSNet/blob/2ced987576a98dc16c5a580839b657948644d44d/upsnet/dataset/base_dataset.py#L941 https://github.com/uber-research/UPSNet/blob/2ced987576a98dc16c5a580839b657948644d44d/upsnet/dataset/base_dataset.py#L942 In this place, the small pictures will be padded 255.

For the second question, you can check the picture of id 69816.

YuwenXiong commented 5 years ago

Thanks for pointing it out, I confirm that these two issues exist. The first one will generate wrong panoptic gt when image shape cannot be divided by 32 (which does happen in coco) and will give slightly wrong supervision for panoptic loss. I'll fix it now. The second one is a little bit tricky, it comes from the bad instance annotation quality of coco. A potential solution is recalculating instance masks from official panoptic gt instead of get masks from original instance annotation.