I have followed the README in main directory to install this project. But when I start to preprocess iwslt dataset following instructions in machine_translation/README.md, the system raises segmentation fault error.
Below is part of messages:
preprocess-iwslt.sh: line 5: 394680 Segmentation fault (core dumped) PYTHONPATH=. python fairseq_cli/preprocess.py --source-lang de --target-lang en --trainpref $TEXT/train --validpref $TEXT/valid --testpref $TEXT/test --destdir data-bin/iwslt14.tokenized.de-en --workers 20
Hi,
I have followed the README in main directory to install this project. But when I start to preprocess iwslt dataset following instructions in machine_translation/README.md, the system raises segmentation fault error.
Below is part of messages:
preprocess-iwslt.sh: line 5: 394680 Segmentation fault (core dumped) PYTHONPATH=. python fairseq_cli/preprocess.py --source-lang de --target-lang en --trainpref $TEXT/train --validpref $TEXT/valid --testpref $TEXT/test --destdir data-bin/iwslt14.tokenized.de-en --workers 20Environment