search

KunpengLi1994 / VSRN

PyTorch code for ICCV'19 paper "Visual Semantic Reasoning for Image-Text Matching"
288 stars 47 forks source link

How to visualize attention? #4

Closed czhxiaohuihui closed 4 years ago

czhxiaohuihui commented 4 years ago

Could you share how to visualize the attention? Vey very Thanks!

KunpengLi1994 commented 4 years ago

Hi,

You can find detailed descriptions about attention visualization in Section 4.5 about Attention visualization of the final image representation.

Basically, we calculate the inner product similarity (same as in Eq. 8) between each region feature V^∗ and the final whole image representation I . Then we rank the image regions V^∗ in the descending order of their correlation with I and assign a score s_i to each v∗i according to its rank r_i. The score is calculate by s_i = λ(k− r_attn)^2 , where k is the total number of regions, λ is a parameter used to emphasize the high ranked regions. We set λ = 50 in our experiments. Then for the final attention map (similarity map), the attention score at each pixel location is obtained by adding up scores of all regions it belongs to.

MI-WENYU commented 2 years ago

Hello, thank you for your great work. I did not find the code of attention visualization in the code you provided, may I ask where it is?