search

nusnlp / mlconvgec2018

Code and model files for the paper: "A Multilayer Convolutional Encoder-Decoder Neural Network for Grammatical Error Correction" (AAAI-18).
GNU General Public License v3.0
185 stars 73 forks source link

Fairseq-py Installation Issue #23

Closed Lizahh closed 5 years ago

Lizahh commented 5 years ago

Hiii!!!!! My error is more related to question asked before in: https://github.com/nusnlp/mlconvgec2018/issues/14#issuecomment-440247589 I have pytorch 0.4.1 and trying to run python setup.py install or python setup.py build both for fairseq-py but getting the same issue in both of them. My pytorch installed is cuda enabled as you suggested in that mentioned link... image I am so stuck at this since more than one week.. Can you please tell me what wrong I am doing. My log file for python setup.py install is : log.txt and for python setup.py build is: log.txt

shamilcm commented 5 years ago

We have tested our master branch only with Python3.5 (Uses a fork of an old version of Fairseq https://github.com/shamilcm/fairseq-py which requires PyTorch 0.2.0post3 compiled from source)

We have a newer branch (https://github.com/nusnlp/mlconvgec2018/tree/fairseq0.5) which runs on Fairseq 0.5 with Pytorch 0.4.1. However, it was tested on Python3.6. I suggest using the same Python3.6 version (instead of Python3.7), Pytorch version and Fairseq version.

Lizahh commented 5 years ago

I have used newer branch with fairseq-0.5 and its working completely fine .. Thanks for helping

Lizahh commented 5 years ago

Thanks Shamil Chollampatt. This really helped a lot. I am done with training of the model, but been stuck at testing again. I'm not so sure though if I should post my issue of testing on main repository as my mentioned issue was closed after being resolved. By, thinking so, I may ask at here. When I try to test the model, the output is like this:

++ source paths.sh +++++ dirname paths.sh ++++ cd . ++++ pwd +++ BASE_DIR=/home/aliza/project/mlconvgec2018 +++ DATA_DIR=/home/aliza/project/mlconvgec2018/data +++ MODEL_DIR=/home/aliza/project/mlconvgec2018/models +++ SCRIPTS_DIR=/home/aliza/project/mlconvgec2018/scripts +++ SOFTWARE_DIR=/home/aliza/project/mlconvgec2018/software ++ '[' 4 -ge 4 ']' ++ input_file=data/test/conll14st-test/conll14st-test.tok.src ++ output_dir=outputs_embed_eolm ++ device=5 ++ model_path=models/mlconv_embed ++ '[' 4 -eq 6 ']' ++ [[ -d models/mlconv_embed ]] +++ ls models/mlconv_embed/model1.pt models/mlconv_embed/model2.pt models/mlconv_embed/model3.pt models/mlconv_embed/model4.pt +++ tr '\n' ' ' +++ sed 's| ([^$])|:\1|g' ++ models='models/mlconv_embed/model1.pt:models/mlconv_embed/model2.pt: models/mlconv_embed/model3.pt:models/mlconv_embed/model4.pt ' ++ echo models/mlconv_embed/model1.pt:models/mlconv_embed/model2.pt: models/mlconv_embed/model3.pt:models/mlconv_embed/model4.pt models/mlconv_embed/model1.pt:models/mlconv_embed/model2.pt: models/mlconv_embed/model3.pt:models/mlconv_embed/model4.pt ++ FAIRSEQPY=/home/aliza/project/mlconvgec2018/software/fairseq-py ++ NBEST_RERANKER=/home/aliza/project/mlconvgec2018/software/nbest-reranker ++ beam=12 ++ nbest=12 ++ threads=12 ++ mkdir -p outputs_embed_eolm ++ python2.7 /home/aliza/project/mlconvgec2018/scripts/apply_bpe.py -c /home/aliza/project/mlconvgec2018/models/bpe_model/train.bpe.model ++ CUDA_VISIBLE_DEVICES=5 ++ python3.6 /home/aliza/project/mlconvgec2018/software/fairseq-py/interactive.py --no-progress-bar --path models/mlconv_embed/model1.pt: models/mlconv_embed/model2.pt:models/mlconv_embed/model3.pt: models/mlconv_embed/model4.pt --beam 12 --nbest 12 /home/aliza/project/mlconvgec2018/models/data_bin --source-lang src --target-lang trg Traceback (most recent call last): File "/home/aliza/project/mlconvgec2018/software/fairseq-py/interactive.py", line 162, in main(args) File "/home/aliza/project/mlconvgec2018/software/fairseq-py/interactive.py", line 77, in main models, model_args = utils.load_ensemble_for_inference(model_paths, task) File "/home/aliza/project/mlconvgec2018/software/fairseq-py/fairseq/utils.py", line 156, in load_ensemble_for_inference model = task.build_model(args) File "/home/aliza/project/mlconvgec2018/software/fairseq-py/fairseq/tasks/fairseq_task.py", line 43, in build_model return models.build_model(args, self) File "/home/aliza/project/mlconvgec2018/software/fairseq-py/fairseq/models/init.py", line 25, in build_model return ARCH_MODEL_REGISTRY[args.arch].build_model(args, task) File "/home/aliza/project/mlconvgec2018/software/fairseq-py/fairseq/models/fconv.py", line 67, in build_model encoder_embed_dict = utils.parse_embedding(args.encoder_embed_path) File "/home/aliza/project/mlconvgec2018/software/fairseq-py/fairseq/utils.py", line 256, in parse_embedding with open(embed_path) as f_embed: FileNotFoundError: [Errno 2] No such file or directory: '/home.local/shamil/wiki/wiki.bpe.fasttext/model.vec'

As onced someone was having the same issue in old repository and you suggested to change the fairseq/utils.py. Are there some hardcoded path in this new fairseq-0.5 also?? Am I supposed to make some alteration in its fairseq/utils.py as well? Thanks

On Sat, May 25, 2019 at 10:07 PM Shamil Chollampatt < notifications@github.com> wrote:

Closed #23 https://github.com/nusnlp/mlconvgec2018/issues/23.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/nusnlp/mlconvgec2018/issues/23?email_source=notifications&email_token=AKT74OJSHV2ZC53SJOTEIL3PXFBYJA5CNFSM4HNR56Y2YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGORUNQIHA#event-2367357980, or mute the thread https://github.com/notifications/unsubscribe-auth/AKT74OOJQ7VZ5COLCJRZCHTPXFBYJANCNFSM4HNR56YQ .

shamilcm commented 5 years ago

Yes, you might have to apply a similar fix (See this commit) https://github.com/shamilcm/fairseq-py/commit/a3c7508610ebc5cb72ee349cbd6a4d183426608d

Lizahh commented 5 years ago

Thanksss.. It worked (Y)