shenxiangzhuang
commented
7 months ago - https://www.pinecone.io/learn/batch-layer-normalization/
- https://stats.stackexchange.com/questions/474440/why-do-transformers-use-layer-norm-instead-of-batch-norm
- https://arxiv.org/pdf/2201.03545.pdf
- https://proceedings.mlr.press/v119/shen20e/shen20e.pdf