About Model Figure?

[yjyGo]

@policeme Hi，In the model figure, 1.Multi-head Attention Distribution part: what's '1 2 3 ... k-2 k-1 k'? Do they represent different head attention？ 2.Final Distribution: Why is Vocabulary distribution on top, copy distribution below?

[xiongma]

1. first question, I don't understand u question, for my opinion, different head attention means different feature, in the attention final, we concat them.
2. second question, I don't understand, first, I don't concat the vocabulary distribution and attention, just add them by vocab index.

[yjyGo]

[yjyGo]

@policeme look the picture above, especially the content of red circle, you will understand my questions.

[xiongma]

not concat.just add. See function final_dist

from Alimail iPhone ------------------Original Mail ------------------ From:yjyGo notifications@github.com Date:2019-07-18 20:50:24 Recipient:policeme/transformer-pointer-generator transformer-pointer-generator@noreply.github.com CC:Cally mx15025700935@aliyun.com, Mention mention@noreply.github.com Subject:Re: [policeme/transformer-pointer-generator] About Model Figure? (#12)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[yjyGo]

can I add you wechat number?

[yjyGo]

我的意思是你的最终分布图形里边蓝色在上，红色在下，蓝色表示词汇表分布，红色代表拷贝分布，我可不可以把它颠倒过来，红在下蓝在上，只是单纯的画图而已。

[xiongma]

只是简单的画图，实际是想加，可以看代码，一步步调试，你就会知道。

from Alimail iPhone ------------------Original Mail ------------------ From:yjyGo notifications@github.com Date:2019-07-18 20:55:32 Recipient:policeme/transformer-pointer-generator transformer-pointer-generator@noreply.github.com CC:Cally mx15025700935@aliyun.com, Mention mention@noreply.github.com Subject:Re: [policeme/transformer-pointer-generator] About Model Figure? (#12) 我的意思是你的最终分布图形里边蓝色在上，红色在下，蓝色表示词汇表分布，红色代表拷贝分布，我可不可以把它颠倒过来，红在下蓝在上，只是单纯的画图而已。 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[yjyGo]

多头注意力分布图形的每个数字表示什么

[yjyGo]

我知道相加，论文里也有，你没明白我的意思

[xiongma]

详细请阅读 All Attention Is You Need

from Alimail iPhone ------------------Original Mail ------------------ From:yjyGo notifications@github.com Date:2019-07-18 20:56:54 Recipient:policeme/transformer-pointer-generator transformer-pointer-generator@noreply.github.com CC:Cally mx15025700935@aliyun.com, Mention mention@noreply.github.com Subject:Re: [policeme/transformer-pointer-generator] About Model Figure? (#12) 多头注意力分布图形的每个数字表示什么 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[xiongma]

你没描述清楚，请详细描述问题

from Alimail iPhone ------------------Original Mail ------------------ From:yjyGo notifications@github.com Date:2019-07-18 20:57:23 Recipient:policeme/transformer-pointer-generator transformer-pointer-generator@noreply.github.com CC:Cally mx15025700935@aliyun.com, Mention mention@noreply.github.com Subject:Re: [policeme/transformer-pointer-generator] About Model Figure? (#12) 我知道相加，论文里也有，你没明白我的意思 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[yjyGo]

你直接回答我截图里面的那些1,2,3，...,k是不是表示第几个头

[xiongma]

不是，下次问的时候请阅读代码后提问

[yjyGo]

不好意思，言语上表达不清楚很抱歉，可以加微信吗？方便交流。