Open NightMachinery opened 2 years ago
I was getting the import error while running the generate.py script. I simply used the latest stable version (0.10.2) of fairseq using pip install -U fairseq and the error vanished. PS: Earlier I had fairseq v0.9.0 installed.
@aadarshsingh191198 commented on Dec 15, 2021, 1:58 PM GMT+3:30:
I was getting the import error while running the generate.py script. I simply used the latest stable version (0.10.2) of fairseq using pip install -U fairseq and the error vanished.
PS: Earlier I had fairseq v0.9.0 installed.
Can you share a working Colab demo? I get this error:
!cd /content/scitldr && python scripts/generate.py /content/models SciTLDR-Data/SciTLDR-AIC/ctrl /content/models --checkpoint_file bart-xsum.tldr-aic.pt --beam 5 --lenpen 0.8 --test_fname test.hypo
Traceback (most recent call last):
File "scripts/generate.py", line 99, in <module>
generate_TLDRs(**vars(args))
File "scripts/generate.py", line 17, in generate_TLDRs
task='translation'
File "/usr/local/lib/python3.7/dist-packages/fairseq/models/bart/model.py", line 122, in from_pretrained
**kwargs,
File "/usr/local/lib/python3.7/dist-packages/fairseq/hub_utils.py", line 72, in from_pretrained
arg_overrides=kwargs,
File "/usr/local/lib/python3.7/dist-packages/fairseq/checkpoint_utils.py", line 279, in load_model_ensemble_and_task
state = load_checkpoint_to_cpu(filename, arg_overrides)
File "/usr/local/lib/python3.7/dist-packages/fairseq/checkpoint_utils.py", line 225, in load_checkpoint_to_cpu
f, map_location=lambda s, l: default_restore_location(s, "cpu")
File "/usr/local/lib/python3.7/dist-packages/torch/serialization.py", line 529, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/usr/local/lib/python3.7/dist-packages/torch/serialization.py", line 692, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '\x0a'.
(Which I guess is the same issue as https://github.com/allenai/scitldr/issues/11 )
Here is a Colab notebook where I tried to do inference using this model.
loading archive file /content/models loading archive file SciTLDR-Data/SciTLDR-AIC/ctrl-bin Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/fairseq/checkpoint_utils.py", line 151, in load_checkpoint_to_cpu from fairseq.fb_pathmgr import fb_pathmgr ModuleNotFoundError: No module named 'fairseq.fb_pathmgr' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "scripts/generate.py", line 99, in <module> generate_TLDRs(**vars(args)) File "scripts/generate.py", line 17, in generate_TLDRs task='translation' File "/usr/local/lib/python3.7/dist-packages/fairseq/models/bart/model.py", line 104, in from_pretrained **kwargs, File "/usr/local/lib/python3.7/dist-packages/fairseq/hub_utils.py", line 68, in from_pretrained arg_overrides=kwargs, File "/usr/local/lib/python3.7/dist-packages/fairseq/checkpoint_utils.py", line 190, in load_model_ensemble_and_task state = load_checkpoint_to_cpu(filename, arg_overrides) File "/usr/local/lib/python3.7/dist-packages/fairseq/checkpoint_utils.py", line 160, in load_checkpoint_to_cpu path, map_location=lambda s, l: default_restore_location(s, "cpu") File "/usr/local/lib/python3.7/dist-packages/torch/serialization.py", line 529, in load return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args) File "/usr/local/lib/python3.7/dist-packages/torch/serialization.py", line 692, in _legacy_load magic_number = pickle_module.load(f, **pickle_load_args) _pickle.UnpicklingError: invalid load key, '\x0a'.
The problem happens when the checkpoint is downloaded via wget. If you download it to your machine locally, you will notice that the file is much larger and comes under another name: scitldr_bart-xsum.tldr-aic.pt
Here is a Colab notebook where I tried to do inference using this model.