Distilled or smaller pretrained models?

cardiffnlp / xlm-t

Repository for XLM-T, a framework for evaluating multilingual language models on Twitter data

Apache License 2.0

142 stars 23 forks source link

Distilled or smaller pretrained models? #7

Open kaliaanup opened 2 years ago

kaliaanup commented 2 years ago

xlm-t works really well on my dataset however to make it effective performance-wise I was wondering if you guys are planning to release smaller pretrained or distilled models?

Best Anup