No such file or directory: './data-bin/pt-data-0508/dict.txt'

[huberemanuel]

Following the official documentation from fairseq, I tried to load the model with the following command:

from fairseq import checkpoint_utils
checkpoint_utils.load_model_ensemble("tupe_ckp/model.pt")

And, got the following error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/ipeas22062020/projects/LAL-Parser/src_joint/TUPE/fairseq/checkpoint_utils.py", line 156, in load_model_ensemble
    ensemble, args, _task = load_model_ensemble_and_task(filenames, arg_overrides, task)
  File "/home/ipeas22062020/projects/LAL-Parser/src_joint/TUPE/fairseq/checkpoint_utils.py", line 171, in load_model_ensemble_and_task
    task = tasks.setup_task(args)
  File "/home/ipeas22062020/projects/LAL-Parser/src_joint/TUPE/fairseq/tasks/__init__.py", line 17, in setup_task
    return TASK_REGISTRY[args.task].setup_task(args, **kwargs)
  File "/home/ipeas22062020/projects/LAL-Parser/src_joint/TUPE/fairseq/tasks/masked_lm.py", line 70, in setup_task
    dictionary = Dictionary.load(os.path.join(paths[0], 'dict.txt'))
  File "/home/ipeas22062020/projects/LAL-Parser/src_joint/TUPE/fairseq/data/dictionary.py", line 192, in load
    d.add_from_file(f, ignore_utf_errors)
  File "/home/ipeas22062020/projects/LAL-Parser/src_joint/TUPE/fairseq/data/dictionary.py", line 209, in add_from_file
    raise fnfe
  File "/home/ipeas22062020/projects/LAL-Parser/src_joint/TUPE/fairseq/data/dictionary.py", line 203, in add_from_file
    with open(f, 'r', encoding='utf-8') as fd:
FileNotFoundError: [Errno 2] No such file or directory: './data-bin/pt-data-0508/dict.txt'

This file pt-data-0508/dict.txt is different from the already provided dict.txt file?

[guolinke]

We use different data processing pipeline (and model code) as fairseq. you cannot use our ckp in official fairseq.

[guolinke]

we had released dict.txt with model checkpoint: https://guolinke.blob.core.windows.net/tupe/tupe_ckp.tar.gz

[huberemanuel]

Thank you for the response! I am using this repo modified version of fairseq. I was able to load the model altering where fairseq looks for dict.txt in that way:

from fairseq import checkpoint_utils
checkpoint_utils.load_model_ensemble("tupe_ckp/model.pt", {"data": "tupe_ckp"})

Concerning the pre-processing, can I run the scripts already provided to perform it and use the model as I loaded it, or would it be wrong?

[guolinke]

I guess load_model_ensemble will load args part saved in checkpoint. maybe you can try to reset args in tupe_ckp/model.pt.