How to understand attention bias?

MendelXu / SAN

Open-vocabulary Semantic Segmentation

https://mendelxu.github.io/SAN/

MIT License

295 stars 27 forks source link

How to understand attention bias? #23

Closed Haerxu closed 9 months ago

Haerxu commented 1 year ago

Hi,

How to understand the attention bias generated from the adapter? I cannot get its purpose.

MendelXu commented 12 months ago

I think a simplified alternative could be "soft mask", used to direct the attention of the CLIP model. However, it is a little beyond the soft mask as we found it could somewhat improve the original clip model's recognition ability on stuff classes.