search

uber-research / UPSNet

UPSNet: A Unified Panoptic Segmentation Network
Other
645 stars 120 forks source link

How could we get PQ values? #18

Closed WenFuLee closed 5 years ago

WenFuLee commented 5 years ago

No matter I run training or testing script, the metrics all I see are AP or AR. Wondering if we should we calculate PQ values ourselves. Or there is anything I missed? Thanks.

Xiaolongbetter commented 5 years ago

When test on cityscapes, I can get PQ values directly. And I could get the panoptic segmentation results like below： image I think you should describe the details of the problem you are experiencing more carefully.

dingguo1996 commented 5 years ago

@WenFuLee I meet the same problem as you said, but I just test the cocodataset. Furthermore, I found the another problem is that key "color" is not in panoptic _coco_categories_stiff.json after coco_init.sh which seems to cause the problem when we calculate PQ values. When I refer to the panoptic_api in official github: https://github.com/cocodataset/panopticapi/blob/master/panoptic_coco_categories.json It allocates every categories with different colors, which is different from panoptic _coco_categories_stiff.json without the key "color".

Xiaolongbetter commented 5 years ago

@qq237942920 For coco dataset，Please pull the latest code ''init_coco.py'' and regenerate panoptic_val2017_stff.json. I think it will be fine.

dingguo1996 commented 5 years ago

@YuwenXiong I think your team completed the work before the update of panoptic_api, so when I git clone the updated panoptic_api, you still use it with old version. Actually, there are many issues about COCO Panoptic data does not have 'color' attribute in panoptic_api. If I'm wrong about your project, I'm so sorry and looking forward your reply.

dingguo1996 commented 5 years ago

@qq237942920 For coco dataset，Please pull the latest code ''init_coco.py'' and regenerate panoptic_val2017_stff.json. I think it will be fine.

I tried it again ,but it still tell me the same problem.

dingguo1996 commented 5 years ago

@Xiaolongbetter @YuwenXiong Oh,I found the project had updated 4 hours ago, so I think it fixed the bug about it. I'm so sorry for my problem. Thank you for your patient reply.

WenFuLee commented 5 years ago

@Xiaolongbetter I saw your above panoptic segmentation results. Wondering when you got your PQ results, did you experience this issue: https://github.com/uber-research/UPSNet/issues/22. Thanks.

WenFuLee commented 5 years ago

This issue was solved after successfully evaluating the test set.