How to visualize attention logits?

mit-han-lab / streaming-llm

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

https://arxiv.org/abs/2309.17453

MIT License

6.38k stars 355 forks source link

How to visualize attention logits? #82

Closed OStars closed 1 month ago

OStars commented 1 month ago

Thanks for your great job!

I have a slightly off-topic question: Could you please tell me how to visualize attention logits like Figure 2 in the paper? I want to use this method to analyze other problems.

Hopping for your reply!

DerrickYLJ commented 1 month ago

Hi, have you figured out how to visualize the attention logits in the paper?