How to train a multi-lingual model with faieseq?

[michelleqyhqyh]

❓ Questions and Help

What is your question?

I hope to train a multi-lingual model with faieseq. Does the parameter "--task" of the command api "fairseq-train" should be "cross_lingual_lm"? Can anyone give me a detailed codes for this command, please? And how can I prepare my various langauges data for fairseq?

What have you tried?

What's your environment?

• fairseq Version (e.g., 1.0 or main):
• PyTorch Version (e.g., 1.0)
• OS (e.g., Linux): Linux
• How you installed fairseq (pip, source): pip
• Build command you used (if compiling from source):
• Python version: 3.7
• CUDA/cuDNN version:
• GPU models and configuration:
• Any other relevant information:

[gmryu]

First of all, I haven't tried a multi-lingual model and you might already find the answer. In case you haven't, check out the followings:

1. an official example https://github.com/pytorch/fairseq/tree/main/examples/multilingual (or https://github.com/pytorch/fairseq/pull/527/commits/6088dfdc44d3ccfa12c5ce2257a1ba8aca6a7a01 even older)
   you may also find a lot of examples at fairseq/examples/, search your cloned fairseq for "multilingual" with an advanced editor (like vscode) may also help you find the direction.
2. every "task" implementation can be found in fairseq/fairseq/tasks/, every "dataset" implementation is inside fairseq/fairseq/data/. fairseq-train,preprocess is _fairseq/fairseqcli/train.py and preprocess.py. By using an advanced editor(like vscode), you will grasp how a task, a model and a dataset, a dictionray works with each other.