Machine Translation Using Transformers namely T5 and XLM-RoBERTa

[nevasini]

I plan to implement 2 transformer models, and finally choose the one which performs the best on evaluation. T5 (Text-to-Text Transfer Transformer) is a powerful and flexible Transformer-based language model that can be fine-tuned for a variety of natural language processing tasks, including machine translation. Unlike traditional machine translation models that are designed specifically for translation, T5 can be fine-tuned on a diverse set of text-to-text tasks, including translation that we desire to work on. Whereas XLM-RoBERTa (Cross-lingual Language Model RoBERTa) https://arxiv.org/pdf/1911.02116.pdf is a state-of-the-art multilingual language model developed by Facebook AI. It is based on the RoBERTa architecture, which is a variant of the BERT model that has been pre-trained on a large corpus of unlabeled text data using a masked language modeling objective. The key innovation of XLM-RoBERTa is its ability to perform cross-lingual language modeling, which means it can learn representations of multiple languages simultaneously

[vikasgrewal16]

can i get to know something about this issue ? @nevasini