WIP: UL2 merge

[RaymondLi0]

This PR is based on https://github.com/NVIDIA/Megatron-LM/pull/268 In addition:

• changed the variance of the masked-span length to scale with the mean
• truncate the sequences after the masking in the decoder-only case.
• some slight refactor

TODO: getting around 30%reduced throughput with UL2.