Investigate the cross-lingual transferability of monolingual BERT representations to Irish

[jbrry]

Another baseline to consider: use a monolingual BERT model and investigate methods for transferring it to Irish, see e.g. Artetxe et al. 2020 On the Cross-lingual Transferability of Monolingual Representations.

@fosterjen, this might be a fun project for some of your students who want to work on Irish NLP.

[jbrry]

See also Tran 2020 From English to Foreign Languages: Transferring Pretrained Language Models, and (code link).

[jowagner]

Also:

• Li et al. 2021 Cross-lingual Transferring of Pre-trained Contextualized Language Models
• Sun et al. 2021 Cross-lingual Universal Dependency Parsing Only from One Monolingual Treebank