Attention Map Visualization Script in "Vision Transformers Need Registers" Paper

facebookresearch / dinov2

PyTorch code and models for the DINOv2 self-supervised learning method.

Apache License 2.0

8.92k stars 784 forks source link

Attention Map Visualization Script in "Vision Transformers Need Registers" Paper #285

Open JihwanEom opened 11 months ago

JihwanEom commented 11 months ago

Hi there, thanks for merging #282!

I was wondering if you could release the script you used for visualizing the attention maps in the VISION TRANSFORMERS NEED REGISTERS paper. It caught my eye, and I'm keen on diving into it a bit more.

Appreciate it!

mileseverett commented 11 months ago

Would also love this

AriesChen-UPC commented 11 months ago

Using #90, you can generate attention maps of the dinov2 version, but it is unable to generate attention maps of the dinov2 with registers version. Looking forward to an update for this code