facebookresearch / fairseq

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
MIT License
30.64k stars 6.42k forks source link

How to train a multi-lingual model with faieseq? #4296

Open michelleqyhqyh opened 2 years ago

michelleqyhqyh commented 2 years ago

❓ Questions and Help

What is your question?

I hope to train a multi-lingual model with faieseq. Does the parameter "--task" of the command api "fairseq-train" should be "cross_lingual_lm"? Can anyone give me a detailed codes for this command, please? And how can I prepare my various langauges data for fairseq?

What have you tried?

What's your environment?

gmryu commented 2 years ago

First of all, I haven't tried a multi-lingual model and you might already find the answer. In case you haven't, check out the followings:

  1. an official example https://github.com/pytorch/fairseq/tree/main/examples/multilingual (or https://github.com/pytorch/fairseq/pull/527/commits/6088dfdc44d3ccfa12c5ce2257a1ba8aca6a7a01 even older)
    you may also find a lot of examples at fairseq/examples/, search your cloned fairseq for "multilingual" with an advanced editor (like vscode) may also help you find the direction.
  2. every "task" implementation can be found in fairseq/fairseq/tasks/, every "dataset" implementation is inside fairseq/fairseq/data/. fairseq-train,preprocess is _fairseq/fairseqcli/train.py and preprocess.py. By using an advanced editor(like vscode), you will grasp how a task, a model and a dataset, a dictionray works with each other.