Feature request: Visualize attention rollout from the output to the input

rail-berkeley / crossformer

MIT License

185 stars 17 forks source link

Feature request: Visualize attention rollout from the output to the input #6

Open jsll opened 3 weeks ago

jsll commented 3 weeks ago

Hi,

I am trying to implement the attention rollout for the CrossFormer to visualize the attention from the output token to the input space. Please take a look at the image below. Do you think this feature is something that you could consider adding?