tfzhou / ProtoSeg

CVPR2022 (Oral) - Rethinking Semantic Segmentation: A Prototype View
https://arxiv.org/abs/2203.15102
MIT License
346 stars 40 forks source link

Questions about K prototypes #9

Open RenLibo-aircas opened 2 years ago

RenLibo-aircas commented 2 years ago

Hi, I'm interested in your work. I wonder if there are k prototypes of some classes that become similar after training? For example, after visualization according to Figure 3 in the paper, it will be found that the activation area of each prototype is roughly the same. I found this problem while running your code. I suspect it's caused by some classes of my dataset that don't have meaningful parts.

Hope to receive your reply, thanks!

wenguanwang commented 2 years ago

@RenLibo-aircas thanks for your interest. Basically, the equipartition constraint (see Eq. 8) in Sinkhorn-Knopp iteration can improve the diversity of the cluster centers. But in practice, case-by-case consideration is needed. For example, if the patterns of the semantic classes of your interest are simple or the number of training data is limited, there is no need to adopt a large number of prototypes.