Visualization of weighted attention and correlations of labels

[KongMingxi]

Hi, Thanks a lot for your code and your kind reply. I'm interested in the visualization of weighted attention and the correlation between neuron activations and label locations which is visualized in section 4.3.

Fig 3 shows some examples of attention maps weighted by sigmoid(S) in Eq.(8). Are these attention maps from the layer 'comb.comb_Eltwise_0'? How can I visualize these attention maps corresponding to a specific label?

As shown in Fig 5, the learned neurons from the conv4 layer of SRN are visualized for illustrating the spatial correlations of label locations. How can I visualize the specific relative position patterns of several labels as visualized in section 4.3?

Could you please release a demo for these two questions? Thank you in advance!

[peri044]

Hello @zhufengx , Thanks a lot for the code and details. I have the same question as @KongMingxi . Could you please let me know which layer was used to produce the attention maps? Thank you !!

[zhufengx]

Hello @zhufengx , Thanks a lot for the code and details. I have the same question as @KongMingxi . Could you please let me know which layer was used to produce the attention maps? Thank you !!

Hi, @peri044 , sorry for the inconvenience, I can give more details as follows: (1) Attention map "A" in Eq. (8) is the raw attention map, which corresponds to the yellow matrix "Attenton Map: A" in Fig 2. It is the output of "att.att_Reshape_6" layer in our released "prototxt" file. It is visualized in "Attention" columns in Fig. 3. (2) "U" in Eq. (8) is the weighted attention map. It is the output of "comb.comb_Eltwise_0 " layer in our released "prototxt" file. It is visualized in "Weighted Attention" columns in Fig. 3.

[peri044]

Hello @zhufengx. Thank you for the quick response. Great work. One final question. The label attention maps shown in Fig 1. correspond to MSCOCO dataset right ? In general, how was the quality of label attention maps corresponding to MSCOCO? Do you see them well aligned with regions in the image for most images? Thank you !!

[zhufengx]

Hello @zhufengx. Thank you for the quick response. Great work. One final question. The label attention maps shown in Fig 1. correspond to MSCOCO dataset right ? In general, how was the quality of label attention maps corresponding to MSCOCO? Do you see them well aligned with regions in the image for most images? Thank you !!

Hi, @peri044 , in my observation, the learned attenton maps are well-aligned with image regions in most images.