search

tensorflow / tensor2tensor

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.
Apache License 2.0
15.5k stars 3.49k forks source link

Getting attention weights during Inference #1044

Open sugeeth14 opened 6 years ago

sugeeth14 commented 6 years ago

Hello, I have trained the model given in the walk-though for English-German translation. I was trying to access attention weights but I found there is no direct command or way to access it. So I tried using the code given in https://colab.research.google.com/github/tensorflow/tensor2tensor/blob/master/tensor2tensor/notebooks/hello_t2t.ipynb#scrollTo=OJKU36QAfqOC but found that there are following major differences between the way decoding is done in the walkthrough compared to the above link 1.Walk through has T2T model set to "INFER" whereas the code given in link has mode "EVAL" 2.The beam size and alpha values can be set to a value for decoding in the walkthough case but the same was not done in the link.

I was trying to access the weights as given in the link (using get_att_mats() function) but was not able to set the mode as "INFER" and required beam and alpha values. So is there any way I can get decode results as given in walk-through with the set beam and alpha values and still get the attention weight values. Thanks in advance.

tanndx17 commented 4 years ago

Hi, did you find out how to print the certain layer weight during inference?