Multi-Task Learning for Morphologically rich NMT

Initialization

• Create folders checkpoints and experiments
• Run git submodule update --init --recursive
  • This will clone the Moses SMT toolkit (needed for scripts)

Notes on languages

• Czech
• German
• Estonian
• Finnish
• Inuktitut
• Russian
• Turkish
• Uzbek

Notes on installation and dependencies

Shell scripts make use of:

• GNU parallel (parallel)
• Ripgrep (rg)

uralicNLP

uralicNLP installs argparse which disallows allow_abbrev and will cause fairseq to crash.

To get around this, Uninstall argparse manually after installing everything else from requirements.txt:

pip install -r requirements.txt
pip uninstall argparse

HFST

On both Python 3.8 and Python 3.9, it seems like pip install hfst still fails despite the manylinux version purportedly on PyPi.

Got things to work following this: https://github.com/hfst/hfst/issues/557#issuecomment-881652221