search

xmed-lab / CLIP_Surgery

CLIP Surgery for Better Explainability with Enhancement in Open-Vocabulary Tasks
368 stars 26 forks source link

question for equation 8 #19

Closed zyhhh123 closed 1 year ago

zyhhh123 commented 1 year ago

Thanks for your excellent work, but I can't understand that the redundant features Fr can be obtained with equation 8. Can you help me with this question?

Eli-YiLi commented 1 year ago

Hi,

You can refer to the code: image

We used the class weight w for the multiplied feature Fm. Note, element-wise production with sum is equal to matrix production to calculate the cos similarity. So, Fr is also a kind of weighted mean similarity map. And it's similar to the map from empty string, since most labels are not appeared in one image.