In the source code, is there a LN layer missing in front of IG-MSA？

caiyuanhao1998 / Retinexformer

"Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement" (ICCV 2023) & (NTIRE 2024 Challenge)

https://arxiv.org/abs/2303.06705

MIT License

898 stars 78 forks source link

Closed kairenchen123 closed 1 year ago

caiyuanhao1998 commented 1 year ago

No, another layer normalization is inside the IG-MSA

caiyuanhao1998 commented 1 year ago

In the line 166 - 167 of RetinexFormer_arch.py