keras-team / keras-hub

Pretrained model hub for Keras 3
Apache License 2.0
807 stars 245 forks source link

Add support for returning the attention scores in encoder call. #1644

Open mmilanifard opened 6 months ago

mmilanifard commented 6 months ago

Is your feature request related to a problem? Please describe. Need to extract self attention scores from a call to the encoder block.

Describe the solution you'd like Add "return_attention_scores" arg to the call method of encoder (similar to the signature of the attention layer call). Return self attention scores if indicated.

Describe alternatives you've considered No other way to cleanly extract attention scores IIUC.

Additional context I can create PR myself if it helps. Not sure if this needs to be done on the decoder model in the same PR.

SuryanarayanaY commented 6 months ago

Hi @mmilanifard ,

Please feel free to create a PR. Keras Dev team will review and update if has to be done on decoder side also. Thanks!

anirudhr20 commented 2 months ago

1879