How to visualize the cross-attention map as in Fig.3 in the paper?

XuyangBai / TransFusion

[PyTorch] Official implementation of CVPR2022 paper "TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers". https://arxiv.org/abs/2203.11496

Apache License 2.0

619 stars 76 forks source link

How to visualize the cross-attention map as in Fig.3 in the paper? #41

Open heming7 opened 2 years ago

heming7 commented 2 years ago

I have trained the two stage model on nuscenes dataset and I would like to see how the attention focus on the content. How can I visualize the cross-attention heatmap for a particular scene/image just like figure 3 in the TransFusion paper?

XuyangBai commented 2 years ago

Hi, I just use the matplotlib to draw the returned cross attention map from the MultiheadAttention module as an image

TianlinZhang668 commented 1 year ago

Do you have some related codes? thanks @XuyangBai