ML Attention Luong

[junxnone]

Luong Attention

Reference

• paper - 2014 - Neural Machine Translation by Jointly Learning to Align and Translate
• paper - 2015 - Effective Approaches to Attention-based Neural Machine Translation
• BahdanauAttention与LuongAttention注意力机制简介
• 一文看懂 Bahdanau 和 Luong 两种 Attention 机制的区别

Brief

Name	Pipeline	公式
Global Attention
Local Attention

Alignment Model

Bahdanau Attention vs Luong Attention

Name	Description	score
Bahdanau Attention paper	h(t-1) -> a(t)
Luong Attention paper	h(t) -> a(t)

[junxnone]

junxnone/aiwiki#210