Adding MLM head

[AlexKalinowski-Zoetis]

Will future models be released with MLM heads?

[mkardas]

Hi Alex, all the Galactica models were trained with a Causal LM objective. We don't plan training MLM heads for now.