search

livingingroups / animal2vec

animal2vec: A self-supervised transformer for rare-event raw audio input
MIT License
17 stars 1 forks source link

Unable to load model #2

Open CMGeldenhuys opened 3 days ago

CMGeldenhuys commented 3 days ago

Hi,

I'm trying to load your model weights using the instructions found in the README.

When loading the model I get the following error:

Python 3.9.20 | packaged by conda-forge | (main, Sep 30 2024, 17:49:10)
[GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import nn
im2024-11-28 09:18:16.549653: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
po2024-11-28 09:18:16.743946: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
rt torch2024-11-28 09:18:19.198949: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2024-11-28 09:18:19.199476: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2024-11-28 09:18:19.199487: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
>>> import torch
>>> import numpy as np
>>> from fairseq import checkpoint_utils
>>> path_to_pt_file='/home/cmgeldenhuys/model_store/animal2vec_large_finetuned_MeerKAT_240507.pt'
>>> models, model_args = checkpoint_utils.load_model_ensemble([path_to_pt_file])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/cmgeldenhuys/micromamba/envs/animal2vec/lib/python3.9/site-packages/fairseq/checkpoint_utils.py", line 367, in load_model_ensemble
    ensemble, args, _task = load_model_ensemble_and_task(
  File "/home/cmgeldenhuys/micromamba/envs/animal2vec/lib/python3.9/site-packages/fairseq/checkpoint_utils.py", line 436, in load_model_ensemble_and_task
    task = tasks.setup_task(cfg.task)
  File "/home/cmgeldenhuys/micromamba/envs/animal2vec/lib/python3.9/site-packages/fairseq/tasks/__init__.py", line 39, in setup_task
    cfg = merge_with_parent(dc(), cfg)
  File "/home/cmgeldenhuys/micromamba/envs/animal2vec/lib/python3.9/site-packages/fairseq/dataclass/utils.py", line 500, in merge_with_parent
    merged_cfg = OmegaConf.merge(dc, cfg)
  File "/home/cmgeldenhuys/micromamba/envs/animal2vec/lib/python3.9/site-packages/omegaconf/omegaconf.py", line 321, in merge
    target.merge_with(*others[1:])
  File "/home/cmgeldenhuys/micromamba/envs/animal2vec/lib/python3.9/site-packages/omegaconf/basecontainer.py", line 331, in merge_with
    self._format_and_raise(key=None, value=None, cause=e)
  File "/home/cmgeldenhuys/micromamba/envs/animal2vec/lib/python3.9/site-packages/omegaconf/base.py", line 95, in _format_and_raise
    format_and_raise(
  File "/home/cmgeldenhuys/micromamba/envs/animal2vec/lib/python3.9/site-packages/omegaconf/_utils.py", line 629, in format_and_raise
    _raise(ex, cause)
  File "/home/cmgeldenhuys/micromamba/envs/animal2vec/lib/python3.9/site-packages/omegaconf/_utils.py", line 610, in _raise
    raise ex  # set end OC_CAUSE=1 for full backtrace
  File "/home/cmgeldenhuys/micromamba/envs/animal2vec/lib/python3.9/site-packages/omegaconf/basecontainer.py", line 329, in merge_with
    self._merge_with(*others)
  File "/home/cmgeldenhuys/micromamba/envs/animal2vec/lib/python3.9/site-packages/omegaconf/basecontainer.py", line 347, in _merge_with
    BaseContainer._map_merge(self, other)
  File "/home/cmgeldenhuys/micromamba/envs/animal2vec/lib/python3.9/site-packages/omegaconf/basecontainer.py", line 314, in _map_merge
    dest[key] = src._get_node(key)
  File "/home/cmgeldenhuys/micromamba/envs/animal2vec/lib/python3.9/site-packages/omegaconf/dictconfig.py", line 258, in __setitem__
    self._format_and_raise(
  File "/home/cmgeldenhuys/micromamba/envs/animal2vec/lib/python3.9/site-packages/omegaconf/base.py", line 95, in _format_and_raise
    format_and_raise(
  File "/home/cmgeldenhuys/micromamba/envs/animal2vec/lib/python3.9/site-packages/omegaconf/_utils.py", line 629, in format_and_raise
    _raise(ex, cause)
  File "/home/cmgeldenhuys/micromamba/envs/animal2vec/lib/python3.9/site-packages/omegaconf/_utils.py", line 610, in _raise
    raise ex  # set end OC_CAUSE=1 for full backtrace
omegaconf.errors.ConfigKeyError: Key 'multi_corpus_keys' not in 'AudioConfigCCAS'
    full_key: multi_corpus_keys
    reference_type=Optional[AudioConfigCCAS]
    object_type=AudioConfigCCAS

Any suggestions to what may be causing the issue?

https://github.com/facebookresearch/fairseq/issues/5282 might be related?

Python dependencies ``` ... pip 24.0 pyhd8ed1ab_0 conda-forge python 3.9.20 h13acc7a_1_cpython conda-forge ``` ``` Package Version ---------------------------- ----------- absl-py 2.1.0 antlr4-python3-runtime 4.8 astunparse 1.6.3 audioread 3.0.1 bitarray 3.0.0 cachetools 5.5.0 certifi 2024.8.30 cffi 1.17.1 charset-normalizer 3.4.0 colorama 0.4.6 contourpy 1.3.0 cycler 0.12.1 Cython 3.0.11 decorator 5.1.1 fairseq 0.12.2 # 920a548ca770fb1a951f7f4289b4d3a0c1bc226f filelock 3.16.1 flatbuffers 24.3.25 fonttools 4.55.0 fsspec 2024.10.0 gast 0.4.0 google-auth 2.36.0 google-auth-oauthlib 0.4.6 google-pasta 0.2.0 grpcio 1.68.0 h5py 3.7.0 huggingface-hub 0.26.2 hydra-core 1.0.7 idna 3.10 imageio 2.36.0 importlib_metadata 8.5.0 intervaltree 3.1.0 iopath 0.1.10 iterative-stratification 0.1.7 joblib 1.4.2 keras 2.11.0 kiwisolver 1.4.7 lazy_loader 0.4 libclang 18.1.1 librosa 0.10.1 llvmlite 0.43.0 lxml 5.3.0 Markdown 3.7 MarkupSafe 3.0.2 matplotlib 3.6.3 msgpack 1.1.0 networkx 3.2.1 numba 0.60.0 numpy 1.23.5 nvidia-cublas-cu11 11.10.3.66 nvidia-cuda-nvrtc-cu11 11.7.99 nvidia-cuda-runtime-cu11 11.7.99 nvidia-cudnn-cu11 8.5.0.96 oauthlib 3.2.2 omegaconf 2.0.6 opt_einsum 3.4.0 packaging 24.0 pandas 1.5.3 pillow 11.0.0 pip 24.0 platformdirs 4.3.6 pooch 1.8.2 portalocker 3.0.0 protobuf 3.19.6 pyarrow 15.0.2 pyasn1 0.6.1 pyasn1_modules 0.4.1 pycparser 2.22 pyparsing 3.2.0 python-dateutil 2.9.0.post0 pytz 2024.2 PyYAML 6.0.2 regex 2024.11.6 requests 2.32.3 requests-oauthlib 2.0.0 rsa 4.9 sacrebleu 2.4.3 scikit-image 0.24.0 # Had to change depency as 0.23.1 would not resolve scikit-learn 1.2.0 scipy 1.10.0 seaborn 0.13.2 setuptools 75.6.0 six 1.16.0 sortedcontainers 2.4.0 soundfile 0.12.1 soxr 0.5.0.post1 tabulate 0.9.0 tensorboard 2.11.2 tensorboard-data-server 0.6.1 tensorboard-plugin-wit 1.8.1 tensorflow 2.11.0 tensorflow-estimator 2.11.0 tensorflow-io-gcs-filesystem 0.37.1 termcolor 2.5.0 threadpoolctl 3.5.0 tifffile 2024.8.30 timm 0.6.12 torch 1.13.1 torchaudio 0.13.1 torchvision 0.14.1 tqdm 4.64.1 typing_extensions 4.12.2 urllib3 2.2.3 Werkzeug 3.1.3 wheel 0.45.1 wrapt 1.17.0 zipp 3.21.0 ```
CMGeldenhuys commented 3 days ago

Got it working. Manually uninstall fairseq and then installing from the git repo worked.

pip uninstall fairseq
pip install git+https://github.com/facebookresearch/fairseq.git@920a548ca770fb1a951f7f4289b4d3a0c1bc226f