search

nusnlp / mlconvgec2018

Code and model files for the paper: "A Multilayer Convolutional Encoder-Decoder Neural Network for Grammatical Error Correction" (AAAI-18).
GNU General Public License v3.0
185 stars 73 forks source link

ImportError: cannot import name 'libbleu' #28

Closed Ibuki1129 closed 5 years ago

Ibuki1129 commented 5 years ago

Hi there. When I ran ./run.sh I got an error below. I've been working on this issue for 5days yet I still cannot figure it out. so pls help me!

Here is what I put ./run.sh data/test/conll14st-test/conll14st-test.tok.src outputs 0 models/mlconv/model1.pt

And here is what I got

14st-test.tok.src outputs 0 models/mlconv/model1.pt ++ source paths.sh +++++ dirname paths.sh ++++ cd . ++++ pwd +++ BASE_DIR=/mnt/data/okabe/mlconvgec2018 +++ DATA_DIR=/mnt/data/okabe/mlconvgec2018/data +++ MODEL_DIR=/mnt/data/okabe/mlconvgec2018/models +++ SCRIPTS_DIR=/mnt/data/okabe/mlconvgec2018/scripts +++ SOFTWARE_DIR=/mnt/data/okabe/mlconvgec2018/software ++ '[' 4 -ge 4 ']' ++ input_file=data/test/conll14st-test/conll14st-test.tok.src ++ output_dir=outputs ++ device=0 ++ model_path=models/mlconv/model1.pt ++ '[' 4 -eq 6 ']' ++ [[ -d models/mlconv/model1.pt ]] ++ [[ -f models/mlconv/model1.pt ]] ++ models=models/mlconv/model1.pt ++ FAIRSEQPY=/mnt/data/okabe/mlconvgec2018/software/fairseq-py ++ NBEST_RERANKER=/mnt/data/okabe/mlconvgec2018/software/nbest-reranker ++ beam=12 ++ nbest=12 ++ threads=12 ++ mkdir -p outputs ++ /mnt/data/okabe/mlconvgec2018/scripts/apply_bpe.py -c /mnt/data/okabe/mlconvgec2018/models/bpe_model/train.bpe.model ++ CUDA_VISIBLE_DEVICES=0 ++ python /mnt/data/okabe/mlconvgec2018/software/fairseq-py/generate.py --no-progress-bar --path models/mlconv/model1.pt --beam 12 --nbest 12 --interactive --workers 12 /mnt/data/okabe/mlconvgec2018/models/data_bin ERROR: missing libbleu.so. run python setup.py install Traceback (most recent call last): File "/mnt/data/okabe/mlconvgec2018/software/fairseq-py/generate.py", line 13, in from fairseq import bleu, options, utils, tokenizer File "/mnt/data/okabe/mlconvgec2018/software/fairseq-py/fairseq/bleu.py", line 18, in raise e File "/mnt/data/okabe/mlconvgec2018/software/fairseq-py/fairseq/bleu.py", line 14, in from fairseq import libbleu ImportError: cannot import name 'libbleu'

As this error says can't import libbleu, and I assume it is caused by something about fairseq. But as the guideline said I followed steps and installed PyTorch from source so I have no idea what is wrong.

I'll put info just in case, Environmental: miniconda on Linux PyTorch version: 1.3.0 Python version: 3.6.2

Thank you.

shamilcm commented 5 years ago

The original code for the paper used this fork & commit for Fairseq https://github.com/shamilcm/fairseq-py/commit/ceb2f1200c9e5b8bf42a1033e7638d3e8586609a. As written in the README there, it uses PyTorch 0.2 compiled from source. The whole setup is very outdated. You could alternatively use PyTorch 0.4.1 installed normally with Fairseq 0.5, if you use scripts from fairseq0.5 branch of this repo (https://github.com/nusnlp/mlconvgec2018/tree/fairseq0.5)

Ibuki1129 commented 5 years ago

Thanks a lot for your quick and kind reply. I just tried what you told me and the error didn't show up anymore! But now another error came up.

When I ran below ./run.sh ./data/test/conll14st-test/conll14st-test.tok.src ./data/test/conll14st-test/output 0 ./training/models/mlconv/model1000

I got this error ++ source paths.sh +++++ dirname paths.sh ++++ cd . ++++ pwd +++ BASE_DIR=/mnt/data/okabe/mlconvgec2018 +++ DATA_DIR=/mnt/data/okabe/mlconvgec2018/data +++ MODEL_DIR=/mnt/data/okabe/mlconvgec2018/models +++ SCRIPTS_DIR=/mnt/data/okabe/mlconvgec2018/scripts +++ SOFTWARE_DIR=/mnt/data/okabe/mlconvgec2018/software ++ '[' 4 -ge 4 ']' ++ input_file=./data/test/conll14st-test/conll14st-test.tok.src ++ output_dir=./data/test/conll14st-test/output ++ device=0 ++ model_path=./training/models/mlconv/model1000 ++ '[' 4 -eq 6 ']' ++ [[ -d ./training/models/mlconv/model1000 ]] ++ [[ -f ./training/models/mlconv/model1000 ]] ++ [[ ! -e ./training/models/mlconv/model1000 ]] ++ echo 'Model path not found: ./training/models/mlconv/model1000' Model path not found: ./training/models/mlconv/model1000 ++ FAIRSEQPY=/mnt/data/okabe/mlconvgec2018/software/fairseq-py ++ NBEST_RERANKER=/mnt/data/okabe/mlconvgec2018/software/nbest-reranker ++ beam=12 ++ nbest=12 ++ threads=12 ++ mkdir -p ./data/test/conll14st-test/output ++ /mnt/data/okabe/mlconvgec2018/scripts/apply_bpe.py -c /mnt/data/okabe/mlconvgec2018/models/bpe_model/train.bpe.model ++ CUDA_VISIBLE_DEVICES=0 ++ python /mnt/data/okabe/mlconvgec2018/software/fairseq-py/interactive.py --no-progress-bar --path --beam 12 --nbest 12 --interactive --workers 12 /mnt/data/okabe/mlconvgec2018/models/data_bin usage: interactive.py [-h] [--no-progress-bar] [--log-interval N] [--log-format {json,none,simple,tqdm}] [--seed N] [--task TASK] [--skip-invalid-size-inputs-valid-test] [--max-tokens N] [--max-sentences N] [--gen-subset SPLIT] [--num-shards N] [--shard-id ID] [--path FILE] [--remove-bpe [REMOVE_BPE]] [--cpu] [--quiet] [--beam N] [--nbest N] [--max-len-a N] [--max-len-b N] [--min-len N] [--no-early-stop] [--unnormalized] [--no-beamable-mm] [--lenpen LENPEN] [--unkpen UNKPEN] [--replace-unk [REPLACE_UNK]] [--score-reference] [--prefix-size PS] [--sampling] [--sampling-topk PS] [--sampling-temperature N] [--buffer-size N] interactive.py: error: argument --path: expected one argument

I found a similar issue someone had(you replied to him as well) so I used interactive.py instead of using generate.py. Sadly, I still have an error.

Thanks for your cooperate!

shamilcm commented 5 years ago

If you want to do single model evaluation, give the path to the model file, not the directory.

In your earlier run, the path was models/mlconv/model1.pt Check if the file is in the correct path and pass this to run.sh

On Fri, 20 Sep 2019 at 2:36 PM, Ibuki1129 notifications@github.com wrote:

Thanks a lot for your quick and kind reply. I just tried what you told me and the error didn't show up anymore! But now another error came up.

When I ran below ./run.sh ./data/test/conll14st-test/conll14st-test.tok.src ./data/test/conll14st-test/output 0 ./training/models/mlconv/model1000

I got this error ++ source paths.sh +++++ dirname paths.sh ++++ cd . ++++ pwd +++ BASE_DIR=/mnt/data/okabe/mlconvgec2018 +++ DATA_DIR=/mnt/data/okabe/mlconvgec2018/data +++ MODEL_DIR=/mnt/data/okabe/mlconvgec2018/models +++ SCRIPTS_DIR=/mnt/data/okabe/mlconvgec2018/scripts +++ SOFTWARE_DIR=/mnt/data/okabe/mlconvgec2018/software ++ '[' 4 -ge 4 ']' ++ input_file=./data/test/conll14st-test/conll14st-test.tok.src ++ output_dir=./data/test/conll14st-test/output ++ device=0 ++ model_path=./training/models/mlconv/model1000 ++ '[' 4 -eq 6 ']' ++ [[ -d ./training/models/mlconv/model1000 ]] ++ [[ -f ./training/models/mlconv/model1000 ]] ++ [[ ! -e ./training/models/mlconv/model1000 ]] ++ echo 'Model path not found: ./training/models/mlconv/model1000' Model path not found: ./training/models/mlconv/model1000 ++ FAIRSEQPY=/mnt/data/okabe/mlconvgec2018/software/fairseq-py ++ NBEST_RERANKER=/mnt/data/okabe/mlconvgec2018/software/nbest-reranker ++ beam=12 ++ nbest=12 ++ threads=12 ++ mkdir -p ./data/test/conll14st-test/output ++ /mnt/data/okabe/mlconvgec2018/scripts/apply_bpe.py -c /mnt/data/okabe/mlconvgec2018/models/bpe_model/train.bpe.model ++ CUDA_VISIBLE_DEVICES=0 ++ python /mnt/data/okabe/mlconvgec2018/software/fairseq-py/interactive.py --no-progress-bar --path --beam 12 --nbest 12 --interactive --workers 12 /mnt/data/okabe/mlconvgec2018/models/data_bin usage: interactive.py [-h] [--no-progress-bar] [--log-interval N] [--log-format {json,none,simple,tqdm}] [--seed N] [--task TASK] [--skip-invalid-size-inputs-valid-test] [--max-tokens N] [--max-sentences N] [--gen-subset SPLIT] [--num-shards N] [--shard-id ID] [--path FILE] [--remove-bpe [REMOVE_BPE]] [--cpu] [--quiet] [--beam N] [--nbest N] [--max-len-a N] [--max-len-b N] [--min-len N] [--no-early-stop] [--unnormalized] [--no-beamable-mm] [--lenpen LENPEN] [--unkpen UNKPEN] [--replace-unk [REPLACE_UNK]] [--score-reference] [--prefix-size PS] [--sampling] [--sampling-topk PS] [--sampling-temperature N] [--buffer-size N] interactive.py: error: argument --path: expected one argument

I found a similar issue someone had(you replied to him as well) so I used interactive.py instead of using generate.py. Sadly, I still have an error.

Thanks for your cooperate!

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/nusnlp/mlconvgec2018/issues/28?email_source=notifications&email_token=AAE46MHTAGDA3DXS6V7DRD3QKRVQJA5CNFSM4IYKJHIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7FXMKI#issuecomment-533427753, or mute the thread https://github.com/notifications/unsubscribe-auth/AAE46MH3MIS4M2V6DJJVXZLQKRVQJANCNFSM4IYKJHIA .

Ibuki1129 commented 5 years ago

Thank you very much!