a CNN that combines several convolutional, spatial transformer (Jaderberg et al.,

[cwh1981]

2015), ReLU (Nair & Hinton, 2010), local contrast normalization (Jarrett et al., 2009) and max-pooling (Scherer et al., 2010) layers

[cwh1981]

attention

[cwh1981]

self attention

[cwh1981]

encoder / decoder

[cwh1981]

label smoothing : label noisy case
Attention is all you need : https://arxiv.org/pdf/1706.03762.pdf
http://jalammar.github.io/illustrated-transformer https://medium.com/@adityathiruvengadam/transformer-architecture-attention-is-all-you-need-aeccd9f50d09 Neural Machine Translation by jointly learninig to align and translate ( https://arxiv.org/pdf/1409.0473.pdf )

[cwh1981]

Vision Transformer

[cwh1981]

position embedding