How to overlay the attention map to the original image for visualization?

facebookresearch / detr

End-to-End Object Detection with Transformers

Apache License 2.0

13.69k stars 2.47k forks source link

How to overlay the attention map to the original image for visualization? #445

Open GWwangshuo opened 3 years ago

GWwangshuo commented 3 years ago

❓ How to overlay the attention map to the original image

I follow your turtorial and successfully get the attention map of the cross-attention. However, I have no idea how to generate the visualization as shown in Fig. 6. Could you provide more details of how to overlay the attention map to the original image as shown in Fig. 6? I have tried this for few days, but all failed. I will appreciate if it is convenient for you to provide the source code. That will be greatly helpful. Thanks a lot.

htyjers commented 2 years ago

Do you know how to get that now, I also need some help, thank you very much

zhiqwang commented 2 years ago

I guess you can refer to this https://github.com/facebookresearch/detr/issues/275#issuecomment-721622633 .