How can we get attention weights from example sequence and structure?

evolutionaryscale / esm

Other

1.22k stars 137 forks source link

How can we get attention weights from example sequence and structure? #119

Open what-is-what opened 2 weeks ago

what-is-what commented 2 weeks ago

How can we get attention weights from example sequence and structure? There were no arguments to get attention weights in transformer blocks, unlike esm2.

gelnesr commented 3 days ago

also interested in this feature, if available!