search

catherinesyeh / attention-viz

Visualizing query-key interactions in language + vision transformers
http://attentionviz.com/
MIT License
127 stars 17 forks source link

Cluster view of attention heads #44

Open catherinesyeh opened 1 year ago

catherinesyeh commented 1 year ago

Create a "projection of projections" (similar to fig. 6 in What Does Bert Look At?) to help users discover attention heads that may have similar attention patterns. Could be an alternate global view of attention (more clusters than matrix).