Compressive Transforer

[vilmarzti]

https://github.com/labmlai/annotated_deep_learning_paper_implementations

[vilmarzti]

Basic Transformer is done:

Todo's:

• Add LM-model C-transformer
• Implement training routine with reconstruction loss

[vilmarzti]

Compressive Transformer trains now

• [x] Check loss computation of reconstruction-loss
• [x] Append documentation with shapes in forward methods
• [x] See whether to apply Layernorm after self-attention (compare with Transformer-XL implementation and Paper)