[SIIT] Research Multilingual Model

[new5558]

Do a literature review on the Multilingual Autoregressive model pretraining

Area to focus

1. Pretraining techniques that make multilingual training different from monolingual
2. Autoregressive models that are trained on multilingual (ex. Bloom, mGPT, XGLM) and comparisons
3. Tokenization techniques on multilingual datasets/models
4. Challenges and unanswered questions on multilingual autoregressive models
5. Others that seems important

[ArthurMinovsky]

Output: Report of Survey and Presentation