Pretraining transformer models on MIMIC with large setting

[bugface]

we only have MIMIC model using transformer-base models, we need to pretrain large version as well

• bert large
• roberta large
• albert - xxlarege
• xlent large
• longformer large

[bugface]

@nvbic Can you take a look on how to pretrain large bert on mimic?

[nvbic]

I will have a look.

[bugface]

we will update the roberta-large and bert-large mimic pretrained models shortly