Attention Heatmap?

[ngcabang]

How do I create the attention heat map based from Attention model like the image below?

PS: I can access the graph through tfdebug but I don't know what I am searching for

[oahziur]

Hi @ngcabang

You can start with the code for generate the image summary here. That will generate a heat map without sentences on the left and top margin. The image will be displayed in the image section on Tensorboard.

[Hazepusik]

Hi, @ngcabang ! I faced the same problem now. Have you solved your issue? Thanks

[ttrouill]

Same here, I would like to get each alignment map for each of the top-k translations when beam search is activated, is this planned any time soon? If not, what should be changed for this to work @oahziur ?

[oahziur]

@ttrouill @nave01314 Actually, I am not sure if the attention image for BeamSearch will be correct out of the box since we traversed the decoding tree in BeamSearchDecoder to get the final results, so I guess we will need to do the similar gather_tree operation in the end. We can revisit this after beam_search and alignment_history are compatible.

@nave01314 Can you print the final_context_state.alignment_history see what's inside the tuple?

[ttrouill]

@oahziur Ok thanks for the precisions, is there an already existing branch working on beam_search and alignment_history compatibility that I can follow/contribute?

[oahziur]

@ttrouill I don't know if there is any existing branch or pull request on this, but I think you can follow this issue to get update if there is any.

[ttrouill]

@oahziur Looks like beam_search and alignment_history compatibility issue will soon be resolved and merged

[DaoD]

@nave01314 the same problem, how do you solve it? I do not use beam search, but I still can't get the alignment.

[JulianZhang]

@oahziur Dear, I can see the attention image by tensorboard now , But there this is only one image for one step, does that mean the image is for all training data?

How can I get image for one given sentences?