search

zhangzx-uiuc / AMR-IE

The code repository for AMR guided joint information extraction model (NAACL-2021).
36 stars 8 forks source link

Problem with pretrained models #1

Closed aritan7 closed 3 years ago

aritan7 commented 3 years ago

Hello. I am having problems running the pretrained model. I have follow the instructions to install the packages including Pytorch 1.6, and use the code given in this repository. I tried to load the pretrained model checkpoint downloaded from Google Drive, but there are some problems:

Here's my conda environment:

# packages in environment at ../amr_general/cenv_x86:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             4.5                       1_gnu  
absl-py                   0.13.0                   pypi_0    pypi
antlr4-python3-runtime    4.8                      pypi_0    pypi
attrs                     21.2.0                   pypi_0    pypi
blas                      1.0                         mkl  
blis                      0.4.1                    pypi_0    pypi
ca-certificates           2021.7.5             h06a4308_1  
catalogue                 1.0.0                    pypi_0    pypi
certifi                   2021.5.30        py38h06a4308_0  
cffi                      1.14.6                   pypi_0    pypi
charset-normalizer        2.0.4                    pypi_0    pypi
click                     8.0.1                    pypi_0    pypi
colorama                  0.4.4                    pypi_0    pypi
cudatoolkit               10.2.89              hfd86e86_1  
cymem                     2.0.5                    pypi_0    pypi
dataclasses               0.6                      pypi_0    pypi
dgl                       0.6.0                    pypi_0    pypi
fairseq                   0.8.0                    pypi_0    pypi
fastbpe                   0.1.0                    pypi_0    pypi
future                    0.18.2                   pypi_0    pypi
hydra-core                1.1.0                    pypi_0    pypi
idna                      3.2                      pypi_0    pypi
importlib-resources       5.2.2                    pypi_0    pypi
intel-openmp              2021.3.0          h06a4308_3350  
joblib                    1.0.1                    pypi_0    pypi
ld_impl_linux-64          2.35.1               h7274673_9  
libffi                    3.3                  he6710b0_2  
libgcc-ng                 9.3.0               h5101ec6_17  
libgomp                   9.3.0               h5101ec6_17  
libstdcxx-ng              9.3.0               hd4cf53a_17  
mkl                       2021.3.0           h06a4308_520  
mkl-service               2.4.0            py38h7f8727e_0  
mkl_fft                   1.3.0            py38h42c9631_2  
mkl_random                1.2.2            py38h51133e4_0  
murmurhash                1.0.5                    pypi_0    pypi
ncurses                   6.2                  he6710b0_1  
networkx                  2.6.2                    pypi_0    pypi
ninja                     1.10.2               hff7bd54_1  
numpy                     1.20.3           py38hf144106_0  
numpy-base                1.20.3           py38h74d4b33_0  
omegaconf                 2.1.0                    pypi_0    pypi
openssl                   1.1.1k               h27cfd23_0  
packaging                 21.0                     pypi_0    pypi
pip                       21.0.1           py38h06a4308_0  
plac                      1.1.3                    pypi_0    pypi
preshed                   3.0.5                    pypi_0    pypi
protobuf                  3.17.3                   pypi_0    pypi
pycparser                 2.20                     pypi_0    pypi
python                    3.8.11          h12debd9_0_cpython  
pytorch                   1.6.0           py3.8_cuda10.2.89_cudnn7.6.5_0    pytorch
pyyaml                    5.4.1                    pypi_0    pypi
readline                  8.1                  h27cfd23_0  
regex                     2021.8.3                 pypi_0    pypi
requests                  2.26.0                   pypi_0    pypi
sacrebleu                 2.0.0                    pypi_0    pypi
scipy                     1.7.1                    pypi_0    pypi
setuptools                52.0.0           py38h06a4308_0  
six                       1.16.0             pyhd3eb1b0_0  
smatch                    1.0.4                    pypi_0    pypi
spacy                     2.2.3                    pypi_0    pypi
sqlite                    3.36.0               hc218d9a_0  
srsly                     1.0.5                    pypi_0    pypi
tabulate                  0.8.9                    pypi_0    pypi
thinc                     7.3.1                    pypi_0    pypi
tk                        8.6.10               hbc83047_0  
tokenizers                0.10.3                   pypi_0    pypi
torch-scatter             2.0.6                    pypi_0    pypi
tqdm                      4.62.0                   pypi_0    pypi
transformers              4.3.0                    pypi_0    pypi
transition-amr-parser     0.3.3rc0                 pypi_0    pypi
urllib3                   1.26.6                   pypi_0    pypi
wasabi                    0.8.2                    pypi_0    pypi
wheel                     0.37.0             pyhd3eb1b0_0  
xz                        5.2.5                h7b6447c_0  
zipp                      3.5.0                    pypi_0    pypi
zlib                      1.2.11               h7b6447c_3

Here's the code:

in_checkpoint = "../amr_general/checkpoint_best.pt"
from transition_amr_parser.parse import AMRParser
parser = AMRParser.from_checkpoint(in_checkpoint)
annotations = parser.parse_sentences([['The', 'boy', 'travels'], ['He', 'visits', 'places']])
print(''.join(annotations[0][0]))
Traceback (most recent call last):
  File "run.py", line 3, in <module>
    parser = AMRParser.from_checkpoint(in_checkpoint)
  File "../transition-amr-parser-master/transition_amr_parser/stack_transformer_amr_parser.py", line 242, in from_checkpoint
    model = load_models(args, task, use_cuda)
  File "../transition-amr-parser-master/transition_amr_parser/stack_transformer_amr_parser.py", line 29, in load_models
    models, _ = checkpoint_utils.load_model_ensemble(
  File "../amr_general/cenv_x86/lib/python3.8/site-packages/fairseq/checkpoint_utils.py", line 155, in load_model_ensemble
    ensemble, args, _task = load_model_ensemble_and_task(filenames, arg_overrides, task)
  File "../amr_general/cenv_x86/lib/python3.8/site-packages/fairseq/checkpoint_utils.py", line 166, in load_model_ensemble_and_task
    state = load_checkpoint_to_cpu(filename, arg_overrides)
  File "../amr_general/cenv_x86/lib/python3.8/site-packages/fairseq/checkpoint_utils.py", line 142, in load_checkpoint_to_cpu
    state = _upgrade_state_dict(state)
  File "../amr_general/cenv_x86/lib/python3.8/site-packages/fairseq/checkpoint_utils.py", line 305, in _upgrade_state_dict
    registry.set_defaults(state['args'], models.ARCH_MODEL_REGISTRY[state['args'].arch])
KeyError: 'stack_transformer_6x6_nopos'

Can you please advise how to fix that? Thank you.

zhangzx-uiuc commented 3 years ago

Hello, thanks for your interest in our work! I've just checked the environments and the models and I can successfully load the model on my ubuntu 18.04 machine. Can you make sure that the model file is fully downloaded and correctly unzipped?

Here are my commands to load the model:

(debug) zixuan11@blender08:/shared/nas/data/m1/zixuan11/amr_parser$ python
Python 3.6.10 |Anaconda, Inc.| (default, May  8 2020, 02:54:21)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from transition_amr_parser.parse import AMRParser
>>> in_dir="./amr_general/checkpoint_best.pt"
>>> parser = AMRParser.from_checkpoint(in_dir)
using GPU for models
Using cache found in /home/zixuan11/.cache/torch/hub/pytorch_fairseq_master
Unable to build Cython components. Please make sure Cython is installed if the torch.hub model you are loading depends on it.
loading archive file http://dl.fbaipublicfiles.com/fairseq/models/roberta.large.tar.gz from cache at /home/zixuan11/.cache/torch/pytorch_fairseq/83e3a689e28e5e4696ecb0bbb05a77355444a5c8a3437e0f736d8a564e80035e.c687083d14776c1979f3f71654febb42f2bb3d9a94ff7ebdfe1ac6748dba89d2
| dictionary: 50264 types
Finished loading models

For your information, here are the detailed packages of my conda env:

# packages in environment at /home/zixuan11/anaconda3/envs/debug:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main
_openmp_mutex             4.5                       1_gnu
antlr4-python3-runtime    4.8                      pypi_0    pypi
backcall                  0.2.0                    pypi_0    pypi
blis                      0.4.1                    pypi_0    pypi
ca-certificates           2021.7.5             h06a4308_1
cached-property           1.5.2                    pypi_0    pypi
catalogue                 1.0.0                    pypi_0    pypi
certifi                   2021.5.30        py36h06a4308_0
cffi                      1.14.6                   pypi_0    pypi
charset-normalizer        2.0.4                    pypi_0    pypi
click                     7.1.2                    pypi_0    pypi
colorama                  0.4.4                    pypi_0    pypi
contextvars               2.4                      pypi_0    pypi
cymem                     2.0.5                    pypi_0    pypi
dataclasses               0.8                      pypi_0    pypi
decorator                 5.0.9                    pypi_0    pypi
en-core-web-sm            2.2.5                    pypi_0    pypi
fairseq                   0.7.2                     dev_0    <develop>
fastbpe                   0.1.0                    pypi_0    pypi
future                    0.18.2                   pypi_0    pypi
h5py                      3.1.0                    pypi_0    pypi
hydra-core                1.1.0                    pypi_0    pypi
idna                      3.2                      pypi_0    pypi
immutables                0.16                     pypi_0    pypi
importlib-metadata        4.6.4                    pypi_0    pypi
importlib-resources       5.2.2                    pypi_0    pypi
ipdb                      0.13.9                   pypi_0    pypi
ipython                   7.16.1                   pypi_0    pypi
ipython-genutils          0.2.0                    pypi_0    pypi
jedi                      0.18.0                   pypi_0    pypi
jinja2                    3.0.1                    pypi_0    pypi
ld_impl_linux-64          2.35.1               h7274673_9
libffi                    3.3                  he6710b0_2
libgcc-ng                 9.3.0               h5101ec6_17
libgomp                   9.3.0               h5101ec6_17
libstdcxx-ng              9.3.0               hd4cf53a_17
markupsafe                2.0.1                    pypi_0    pypi
murmurhash                1.0.5                    pypi_0    pypi
ncurses                   6.2                  he6710b0_1
numpy                     1.19.5                   pypi_0    pypi
omegaconf                 2.1.0                    pypi_0    pypi
openssl                   1.1.1k               h27cfd23_0
packaging                 21.0                     pypi_0    pypi
parso                     0.8.2                    pypi_0    pypi
pathy                     0.6.0                    pypi_0    pypi
pexpect                   4.8.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pip                       21.2.2           py36h06a4308_0
plac                      1.1.3                    pypi_0    pypi
portalocker               2.3.0                    pypi_0    pypi
preshed                   3.0.5                    pypi_0    pypi
prompt-toolkit            3.0.19                   pypi_0    pypi
ptyprocess                0.7.0                    pypi_0    pypi
pycparser                 2.20                     pypi_0    pypi
pydantic                  1.8.2                    pypi_0    pypi
pygments                  2.10.0                   pypi_0    pypi
pyparsing                 2.4.7                    pypi_0    pypi
python                    3.6.10               h7579374_2
pyyaml                    5.4.1                    pypi_0    pypi
readline                  8.1                  h27cfd23_0
regex                     2021.8.3                 pypi_0    pypi
requests                  2.26.0                   pypi_0    pypi
sacrebleu                 2.0.0                    pypi_0    pypi
setuptools                52.0.0           py36h06a4308_0
six                       1.16.0                   pypi_0    pypi
smart-open                5.1.0                    pypi_0    pypi
spacy                     2.2.3                    pypi_0    pypi
spacy-legacy              3.0.8                    pypi_0    pypi
sqlite                    3.36.0               hc218d9a_0
srsly                     1.0.5                    pypi_0    pypi
tabulate                  0.8.9                    pypi_0    pypi
thinc                     7.3.1                    pypi_0    pypi
tk                        8.6.10               hbc83047_0
toml                      0.10.2                   pypi_0    pypi
torch                     1.6.0                    pypi_0    pypi
torch-scatter             1.3.2                    pypi_0    pypi
tqdm                      4.62.1                   pypi_0    pypi
traitlets                 4.3.3                    pypi_0    pypi
transition-amr-parser     0.3.3rc0                  dev_0    <develop>
typer                     0.3.2                    pypi_0    pypi
typing-extensions         3.10.0.0                 pypi_0    pypi
urllib3                   1.26.6                   pypi_0    pypi
wasabi                    0.8.2                    pypi_0    pypi
wcwidth                   0.2.5                    pypi_0    pypi
wheel                     0.37.0             pyhd3eb1b0_0
xz                        5.2.5                h7b6447c_0
zipp                      3.5.0                    pypi_0    pypi
zlib                      1.2.11               h7b6447c_3
aritan7 commented 3 years ago

Thank you for your detailed answer. I couldn't run these packages because I didn't follow the installation instructions carefully. It works fine now.

Thank you.

tingchihc commented 1 year ago

@zhangzx-uiuc @aritan7 hi, I have the same problem as you. My error message is KeyError: 'stack_transformer_6x6_nopos' too. Could you tell me which package can solve this problem?

I only miss this transition-amr-parser 0.3.3rc0 dev_0 <develop> in my environment. Could you tell me how to install it from this repository? I think the problem is this one. However, I do not know how to install. Could you explain the method to me? My original method is to clone transition-amr-parser in my environment and setup by IBM readme. Then, I put my codes into this folder, but this looks cannot fix my problem.

thanks,

xiehou-design commented 1 year ago

@zhangzx-uiuc @aritan7 hi, I have the same problem as you. My error message is KeyError: 'stack_transformer_6x6_nopos' too. Could you tell me which package can solve this problem?

I only miss this transition-amr-parser 0.3.3rc0 dev_0 <develop> in my environment. Could you tell me how to install it from this repository? I think the problem is this one. However, I do not know how to install. Could you explain the method to me? My original method is to clone transition-amr-parser in my environment and setup by IBM readme. Then, I put my codes into this folder, but this looks cannot fix my problem.

thanks,

Can you solve the problem? I also meet it. Wait your reply!