TransformerLensOrg / CircuitsVis

Mechanistic Interpretability Visualizations using React
https://alan-cooney.github.io/CircuitsVis/
MIT License
192 stars 29 forks source link

Attention pattern visualization tweaks #54

Closed dkamm closed 1 year ago

dkamm commented 1 year ago

Hey @alan-cooney! I made some quality of life tweaks for the attention pattern visualization

  1. Add a background color for the upper triangular part so I can immediately see the irrelevant part
  2. Reformat the tooltip a bit to separate the index from the token
  3. Force tooltip to be above current cell. Reasoning is that I want to see the whole row and forcing tooltip on left or right gets in the way.
    • (Forcing above still covers previous rows though so maybe the ideal solution is to let the user toggle the direction)
  4. Make the index a suffix instead of a prefix. Indices are aligned and looks better imo
  5. Allow rotation up to 90 deg on the x axis labels. I found forcing 90 deg rotation to be unpleasant
Screenshot 2023-03-10 at 4 05 54 PM