search

pmeletis / panoptic_parts

This repository contains code and tools for reading, processing, evaluating on, and visualizing Panoptic Parts datasets. Moreover, it contains code for reproducing our CVPR 2021 paper results.
https://panoptic-parts.readthedocs.io/en/stable
Apache License 2.0
100 stars 16 forks source link

i can not find this file? cpp_iouparts_24_evalspec.yaml #20

Closed xushilin1 closed 3 years ago

xushilin1 commented 3 years ago

I want to compute partIOU using experimental_eval_PartIOU.py; But I can't find cpp_iouparts_24_evalspec.yaml file.

DdeGeus commented 3 years ago

Hi @xushilin1, thanks for your interest in our work!

Unfortunately, our code currently does not fully support part-level mean IOU computation to calculate the mean IOU scores as reported in our CVPR paper. To be able to do this, you need to take a few more steps.

  1. First, you need to convert the sids_pids (for both the ground-truth and your predictions) to the part_ids used for training part segmentation. These mappings can be found in the eval_spec for PartPQ evaluation: ppq_cpp_19_23_cvpr21_default_evalspec.yaml or ppq_cpp_19_23_cvpr21_grouped_evalspec.yaml, depending on whether you group similar parts.
  2. Using these labels, you can then use a generic mean IOU calculator (there are many on Github) to calculate the mean IOU over all the classes, including the background.
  3. During evaluation, make sure that the pixels for which there is a scene-level class with parts (i.e., sid is in eval_spec.eval_sid_parts), but no part class is labeled, should be ignored during evaluation. This also holds for the pixels for which no valid scene class is provided (i.e., sid is not in eval_spec.eval_sid_total).

If you follow this, you will get part-level mean IOU scores that are comparable with the ones reported in our paper. Please let me know if this is clear, or if you have any follow-up questions.