Is your feature request related to a problem? Please describe.
Need to extract self attention scores from a call to the encoder block.
Describe the solution you'd like
Add "return_attention_scores" arg to the call method of encoder (similar to the signature of the attention layer call). Return self attention scores if indicated.
Describe alternatives you've considered
No other way to cleanly extract attention scores IIUC.
Additional context
I can create PR myself if it helps. Not sure if this needs to be done on the decoder model in the same PR.
Is your feature request related to a problem? Please describe. Need to extract self attention scores from a call to the encoder block.
Describe the solution you'd like Add "return_attention_scores" arg to the call method of encoder (similar to the signature of the attention layer call). Return self attention scores if indicated.
Describe alternatives you've considered No other way to cleanly extract attention scores IIUC.
Additional context I can create PR myself if it helps. Not sure if this needs to be done on the decoder model in the same PR.