search

pytorch / translate

Translate - a PyTorch Language Library
BSD 3-Clause "New" or "Revised" License
824 stars 192 forks source link

While running pretrained model(IWSLT 2014) , observed below errors #593

Closed streamhsa closed 5 years ago

streamhsa commented 5 years ago

Traceback (most recent call last): File "pytorch_translate/generate.py", line 705, in main() File "pytorch_translate/generate.py", line 609, in main generate(args) File "pytorch_translate/generate.py", line 634, in generate args.path.split(":") File "/root/pytorch/fairseq/pytorch-translate/translate/pytorch_translate/utils.py", line 116, in load_diverse_ensemble_for_inference task = tasks.setup_task(checkpoints_data[0]["args"]) File "/root/pytorch/fairseq/fairseq/tasks/init.py", line 19, in setup_task return TASK_REGISTRY[args.task].setup_task(args, **kwargs) File "/root/pytorch/fairseq/pytorch-translate/translate/pytorch_translate/tasks/pytorch_translate_task.py", line 124, in setup_task args.source_vocab_file File "/root/pytorch/fairseq/fairseq/data/dictionary.py", line 184, in load d.add_from_file(f, ignore_utf_errors) File "/root/pytorch/fairseq/fairseq/data/dictionary.py", line 201, in add_from_file raise fnfe File "/root/pytorch/fairseq/fairseq/data/dictionary.py", line 195, in add_from_file with open(f, 'r', encoding='utf-8') as fd: FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints/dictionary-de.txt'

cndn commented 5 years ago

Could you double check your working directory and whether you have the dictionary under the expected path?

leimao commented 5 years ago

I got the same problem. bash pytorch_translate/examples/generate_iwslt14.sh would download a pre-trained model and data containing dictionaries automatically. However, it is looking for files in a path that would be generated from training process.

leimao commented 5 years ago

Even with dictionary-de.txt in the checkpoints, I still got the following error:

Traceback (most recent call last):
  File "pytorch_translate/generate.py", line 715, in <module>
    main()
  File "pytorch_translate/generate.py", line 619, in main
    generate(args)
  File "pytorch_translate/generate.py", line 644, in generate
    args.path.split(":")
  File "/workspace/fb-transformer/translate/pytorch_translate/utils.py", line 118, in load_diverse_ensemble_for_inference
    task = tasks.setup_task(checkpoints_data[0]["args"])
  File "/workspace/fb-transformer/fairseq/fairseq/tasks/__init__.py", line 17, in setup_task
    return TASK_REGISTRY[args.task].setup_task(args, **kwargs)
  File "/workspace/fb-transformer/translate/pytorch_translate/tasks/pytorch_translate_task.py", line 151, in setup_task
    use_char_target = (args.char_target_vocab_file != "") or (
AttributeError: 'Namespace' object has no attribute 'char_target_vocab_file'

My assessment is that the pre-trained model downloaded is no longer compatible with the existing code.

jhcross commented 5 years ago

This should be addressed by https://github.com/pytorch/translate/pull/643