Fused Layer Norm

[xrsrke]

TODO

• [x] Port MixedFusedLayerNorm from megatron
• [x] Pass tests
• [x] Convergence test

[xrsrke]

@NouamaneTazi

Both done. I ran both fused layer norm and fast layer norm for 10k update steps. And here are the convergence results.

• Fast layer norm's convergence wandb link

• Fast layer norm convergence wandb link

[xrsrke]

just reran it again after the new changes