search

lingorX / HieraSeg

CVPR2022 - Deep Hierarchical Semantic Segmentation - A structured, pixel-wise description of visual scenes in terms of the class hierarchy.
262 stars 24 forks source link

Evaluation metrics #4

Open kulkarnikeerti opened 1 year ago

kulkarnikeerti commented 1 year ago

Hi @lingorX @wenguanwang, @tfzhou

Thank you for the great paper.

I have a question regarding the evaluation metrics. The metrics used in this paper is the mIoU.

  1. So this mIoU is calculated for the original leaf nodes or does this also contain the parent nodes (which label_Ids are not present in the dataset)?
  2. If I want to compare the segmentation models without and with the hierarchy, is mIoU appropriate metrics for that? If yes, how does this metrics behave? I mean should this increase if we have a hierarchy?

Thanks in advance

joshmyersdean commented 1 year ago

I am also curious about this.

kulkarnikeerti commented 1 year ago
  1. It calculates only for the leaf nodes and does not consider the parent nodes into account while calculating the mIoU
  2. This current implementation does not include the mIoU calculation for parent nodes, so if you just want to compare the leaf nodes you can use it.