Question about the active query sampling

[Hugo-cell111]

Hi! In your paper the queries are sampled from high-entropy pixels, which means they are full of noise. When calculating the contrastive loss with such noisy queries, the noise will be inevitably introduced in the training process. In your opinion how to solve the problem? Thanks!

[lorenmt]

Hi,

Please note that the only queries are of high entropy. It’s possible that the anchor are of low entropy.

[Hugo-cell111]

yes, you are right. I do mean the queries may be noisy, and when using them in the contrastive loss, mistakes might be introduced. For example, when a pixel is predicted as class A(but the actual class is B), and the positive anchors might be sampled from class A, but in fact the class A should be separated from class B. So I wonder if the queries are noisy, how do we make sure the contrastive loss can provide correct supervision?

[lorenmt]

I understand your concern. But the positive anchors here are mean class representations computed from all pixels predicted from that class. And since in the context of semantic segmentation, most pixels are very easy to be classified, the noisy/wrongly predicted pixels would have not a huge impact on the final positive anchors. Hope that resolves your questions.

[Hugo-cell111]

Yes, I get your point totally. I just wonder if there is another method that can get rid of the noise from pseudo labels. Thank you very much!

[lorenmt]

It's impossible to get rid of all noise, as long as you don't have the ground-truth labels.

[lorenmt]

No further updates.