search

catherinesyeh / attention-viz

Visualizing query-key interactions in language + vision transformers
http://attentionviz.com/
MIT License
122 stars 15 forks source link

Connect tokens in attention view #45

Closed catherinesyeh closed 1 year ago

catherinesyeh commented 1 year ago

Building off #7, would be cool to connect tokens with lines (e.g., weighted by attention values) to show their intersections and contribute novel visualization of attention.

Maybe only draw top 2-3 attention weights for each token

catherinesyeh commented 1 year ago

Should coordinate with renormalization of left attention view

catherinesyeh commented 1 year ago

good for now