search

bcmi / CaGNet-Zero-Shot-Semantic-Segmentation

Code for our ACMMM2020 paper "Context-aware Feature Generation for Zero-shot Semantic Segmentation".
233 stars 14 forks source link

Calculation of evaluation #1

Closed asdnjadsn closed 4 years ago

asdnjadsn commented 4 years ago

It's a great job and it helps me a lot. I would like to know why background should be filtered out when calculating acc or IOU by VOC？, which will lead to a significantly higher ACC and IOU， thank you

zhangxgu commented 4 years ago

@asdnjadsn Thanks for your interests about our work. We ignore the category ''background'' during evaluation mainly because of two reasons as mentioned in our paper: 1. we strictly follow the setting of SPNet, which was accepted by CVPR19. In SPNet, the author did not include this category during evaluation. 2. since our method is based on the GAN trained with the word embedding, the semantic representation of ''background'' is not so meaningful. In voc and context, sky and playground are all annotated as the background. Thus, we can not use only one word vector of ''background'' to generate distinct backgrounds. If we do so, it will harm our generator. Besides, filtering out the background will decrease the iou and acc, since the acc of background is >90%, while some of other categories can only achieve lower performance.