search

lucidrains / enformer-pytorch

Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch
MIT License
434 stars 81 forks source link

error loading enformer package #8

Open cmlakhan opened 2 years ago

cmlakhan commented 2 years ago

I am trying to install the enformer package but seem to be getting the following error:

>>> import torch
>>> from enformer_pytorch import Enformer
Traceback (most recent call last):
  File "/sc/arion/projects/ad-omics/clakhani/conda/envs/enformer_lightning/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 905, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
  File "/sc/arion/projects/ad-omics/clakhani/conda/envs/enformer_lightning/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/sc/arion/projects/ad-omics/clakhani/conda/envs/enformer_lightning/lib/python3.9/site-packages/transformers/modeling_utils.py", line 76, in <module>
    from accelerate import dispatch_model, infer_auto_device_map, init_empty_weights
ImportError: cannot import name 'dispatch_model' from 'accelerate' (/sc/arion/projects/ad-omics/clakhani/conda/envs/enformer_lightning/lib/python3.9/site-packages/accelerate/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/sc/arion/projects/ad-omics/clakhani/conda/envs/enformer_lightning/lib/python3.9/site-packages/enformer_pytorch/__init__.py", line 2, in <module>
    from enformer_pytorch.modeling_enformer import Enformer, SEQUENCE_LENGTH, AttentionPool
  File "/sc/arion/projects/ad-omics/clakhani/conda/envs/enformer_lightning/lib/python3.9/site-packages/enformer_pytorch/modeling_enformer.py", line 14, in <module>
    from transformers import PreTrainedModel
  File "<frozen importlib._bootstrap>", line 1055, in _handle_fromlist
  File "/sc/arion/projects/ad-omics/clakhani/conda/envs/enformer_lightning/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 895, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/sc/arion/projects/ad-omics/clakhani/conda/envs/enformer_lightning/lib/python3.9/site-packages/transformers/utils/import_utils.py", line 907, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.modeling_utils because of the following error (look up to see its traceback):
cannot import name 'dispatch_model' from 'accelerate' (/sc/arion/projects/ad-omics/clakhani/conda/envs/enformer_lightning/lib/python3.9/site-packages/accelerate/__init__.py)

I simply cloned an existing pytorch environment on Conda (using cuda 11.1 and torch 1.10) and then pip installed the hugging face packages and enformer packages

pip install transformers
pip install datasets
pip install accelerate
pip install tokenizers
pip install enformer-pytorch

Any idea why I'm getting this error?

cmlakhan commented 2 years ago

If it helps here are the packages installed in the conda environment

# packages in environment at /sc/arion/projects/ad-omics/clakhani/conda/envs/enformer_lightning:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                      1_llvm    conda-forge
absl-py                   1.0.0              pyhd8ed1ab_0    conda-forge
accelerate                0.5.1                      py_0    fastai
aiohttp                   3.7.0            py39h07f9747_0    conda-forge
alembic                   1.7.7                    pypi_0    pypi
argcomplete               2.0.0                    pypi_0    pypi
argh                      0.26.2                   pypi_0    pypi
arrow                     1.2.2                    pypi_0    pypi
asciitree                 0.3.3                    pypi_0    pypi
async-timeout             3.0.1                   py_1000    conda-forge
attrs                     21.4.0             pyhd8ed1ab_0    conda-forge
autopage                  0.5.0                    pypi_0    pypi
axial-positional-embedding 0.2.1                    pypi_0    pypi
binaryornot               0.4.4                    pypi_0    pypi
blas                      1.0                         mkl    defaults
blinker                   1.4                        py_1    conda-forge
bottleneck                1.3.2            py39hdd57654_1    defaults
brotli                    1.0.9                he6710b0_2    defaults
brotlipy                  0.7.0           py39h3811e60_1003    conda-forge
bzip2                     1.0.8                h7b6447c_0    defaults
c-ares                    1.18.1               h7f98852_0    conda-forge
ca-certificates           2021.10.8            ha878542_0    conda-forge
cachetools                4.2.4              pyhd8ed1ab_0    conda-forge
certifi                   2021.10.8        py39hf3d152e_1    conda-forge
cffi                      1.14.6           py39he32792d_0    conda-forge
chardet                   3.0.4           py39h079e4ff_1008    conda-forge
charset-normalizer        2.0.9              pyhd8ed1ab_0    conda-forge
click                     8.0.3            py39hf3d152e_1    conda-forge
cliff                     3.10.1                   pypi_0    pypi
cmaes                     0.8.2                    pypi_0    pypi
cmd2                      2.4.1                    pypi_0    pypi
colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
coloredlogs               15.0.1                   pypi_0    pypi
colorlog                  6.6.0                    pypi_0    pypi
cookiecutter              1.7.3                    pypi_0    pypi
cryptography              36.0.1           py39h95dcef6_0    conda-forge
cudatoolkit               11.1.74              h6bb024c_0    nvidia
cycler                    0.10.0           py39h06a4308_0    defaults
cython                    0.29.24          py39hdbfa776_0    defaults
cyvcf2                    0.30.14                  pypi_0    pypi
dataclasses               0.8                pyhc8e2a94_3    conda-forge
datasets                  2.3.2                    pypi_0    pypi
dbus                      1.13.18              hb2f20db_0    defaults
deepspeed                 0.5.9                    pypi_0    pypi
deprecation               2.1.0                    pypi_0    pypi
dill                      0.3.5.1                  pypi_0    pypi
einops                    0.4.1                    pypi_0    pypi
enformer-pytorch          0.5.3                    pypi_0    pypi
expat                     2.4.1                h2531618_2    defaults
fairscale                 0.4.4                    pypi_0    pypi
fasteners                 0.17.3                   pypi_0    pypi
ffmpeg                    4.3                  hf484d3e_0    pytorch
filelock                  3.7.1                    pypi_0    pypi
fontconfig                2.13.1               h6c09931_0    defaults
fonttools                 4.25.0             pyhd3eb1b0_0    defaults
freetype                  2.11.0               h70c0345_0    defaults
fsspec                    2021.11.1          pyhd8ed1ab_0    conda-forge
future                    0.18.2           py39hf3d152e_4    conda-forge
gffutils                  0.10.1                   pypi_0    pypi
giflib                    5.2.1                h7b6447c_0    defaults
glib                      2.69.1               h5202010_0    defaults
gmp                       6.2.1                h2531618_2    defaults
gnutls                    3.6.15               he1e5248_0    defaults
google-auth               2.3.3              pyh6c4a22f_0    conda-forge
google-auth-oauthlib      0.4.6              pyhd8ed1ab_0    conda-forge
greenlet                  1.1.2                    pypi_0    pypi
grpcio                    1.33.2           py39h0c26d4f_2    conda-forge
gst-plugins-base          1.14.0               h8213a91_2    defaults
gstreamer                 1.14.0               h28cd5cc_2    defaults
h5py                      3.6.0            py39ha0f2276_0    defaults
hdf5                      1.10.6               hb1b8bf9_0    defaults
hjson                     3.0.2                    pypi_0    pypi
huggingface-hub           0.8.1                    pypi_0    pypi
humanfriendly             10.0                     pypi_0    pypi
icu                       58.2                 he6710b0_3    defaults
idna                      3.1                pyhd3deb0d_0    conda-forge
importlib-metadata        4.10.0           py39hf3d152e_0    conda-forge
intel-openmp              2021.4.0          h06a4308_3561    defaults
jinja2                    3.0.3                    pypi_0    pypi
jinja2-time               0.2.0                    pypi_0    pypi
joblib                    1.1.0              pyhd3eb1b0_0    defaults
jpeg                      9d                   h7f8727e_0    defaults
kipoi                     0.7.4                    pypi_0    pypi
kipoi-conda               0.2.2                    pypi_0    pypi
kipoi-utils               0.7.1                    pypi_0    pypi
kipoiseq                  0.7.1                    pypi_0    pypi
kiwisolver                1.3.1            py39h2531618_0    defaults
krb5                      1.17.2               h926e7f8_0    conda-forge
lame                      3.100                h7b6447c_0    defaults
lcms2                     2.12                 h3be6417_0    defaults
ld_impl_linux-64          2.35.1               h7274673_9    defaults
libcurl                   7.76.1               hc4aaa36_1    conda-forge
libedit                   3.1.20210910         h7f8727e_0    defaults
libev                     4.33                 h516909a_1    conda-forge
libffi                    3.3                  he6710b0_2    defaults
libgcc-ng                 11.2.0              h1d223b6_11    conda-forge
libgfortran-ng            7.3.0                hdf63c60_0    defaults
libiconv                  1.15                 h63c8f33_5    defaults
libidn2                   2.3.2                h7f8727e_0    defaults
libnghttp2                1.46.0               hce63b2e_0    defaults
libpng                    1.6.37               hbc83047_0    defaults
libprotobuf               3.13.0.1             h8b12597_0    conda-forge
libssh2                   1.10.0               ha56f1ee_2    conda-forge
libstdcxx-ng              9.1.0                hdf63c60_0    defaults
libtasn1                  4.16.0               h27cfd23_0    defaults
libtiff                   4.2.0                h85742a9_0    defaults
libunistring              0.9.10               h27cfd23_0    defaults
libuuid                   1.0.3                h7f8727e_2    defaults
libuv                     1.40.0               h7b6447c_0    defaults
libwebp                   1.2.0                h89dd481_0    defaults
libwebp-base              1.2.0                h27cfd23_0    defaults
libxcb                    1.14                 h7b6447c_0    defaults
libxml2                   2.9.10               hb55368b_3    defaults
linear-attention-transformer 0.19.1                   pypi_0    pypi
linformer                 0.2.1                    pypi_0    pypi
llvm-openmp               12.0.1               h4bd325d_1    conda-forge
llvmlite                  0.38.1                   pypi_0    pypi
local-attention           1.4.3                    pypi_0    pypi
lz4-c                     1.9.3                h295c915_1    defaults
mako                      1.2.0                    pypi_0    pypi
markdown                  3.3.6              pyhd8ed1ab_0    conda-forge
markupsafe                2.0.1                    pypi_0    pypi
matplotlib                3.4.3            py39h06a4308_0    defaults
matplotlib-base           3.4.3            py39hbbc1b5f_0    defaults
memory-efficient-attention-pytorch 0.0.17                   pypi_0    pypi
mixture-of-experts        0.2.1                    pypi_0    pypi
mkl                       2021.4.0           h06a4308_640    defaults
mkl-service               2.4.0            py39h7f8727e_0    defaults
mkl_fft                   1.3.1            py39hd3c417c_0    defaults
mkl_random                1.2.2            py39h51133e4_0    defaults
multidict                 5.2.0            py39h3811e60_1    conda-forge
multiprocess              0.70.13                  pypi_0    pypi
munkres                   1.1.4                      py_0    defaults
natsort                   7.1.1              pyhd3eb1b0_0    defaults
ncls                      0.0.63           py39h38f01e4_0    bioconda
ncurses                   6.3                  h7f8727e_2    defaults
nettle                    3.7.3                hbbd107a_1    defaults
ninja                     1.10.2.3                 pypi_0    pypi
numba                     0.55.2                   pypi_0    pypi
numcodecs                 0.9.1                    pypi_0    pypi
numexpr                   2.7.3            py39h22e1b3c_1    defaults
numpy                     1.21.2           py39h20f2e39_0    defaults
numpy-base                1.21.2           py39h79a1101_0    defaults
oauthlib                  3.1.1              pyhd8ed1ab_0    conda-forge
olefile                   0.46               pyhd3eb1b0_0    defaults
openh264                  2.1.0                hd408876_0    defaults
openssl                   1.1.1l               h7f98852_0    conda-forge
optuna                    2.10.0                   pypi_0    pypi
packaging                 21.3               pyhd8ed1ab_0    conda-forge
pandas                    1.3.4            py39h8c16a72_0    defaults
pbr                       5.8.1                    pypi_0    pypi
pcre                      8.45                 h295c915_0    defaults
perceiver-pytorch         0.8.3                    pypi_0    pypi
performer-pytorch         1.1.3                    pypi_0    pypi
pillow                    8.4.0            py39h5aabda8_0    defaults
pip                       21.2.4           py39h06a4308_0    defaults
polars                    0.13.55                  pypi_0    pypi
poyo                      0.5.0                    pypi_0    pypi
prettytable               3.2.0                    pypi_0    pypi
product-key-memory        0.1.10                   pypi_0    pypi
protobuf                  3.13.0.1         py39h41458e0_1    conda-forge
psutil                    5.9.0                    pypi_0    pypi
py-cpuinfo                8.0.0                    pypi_0    pypi
pyarrow                   8.0.0                    pypi_0    pypi
pyasn1                    0.4.8                      py_0    conda-forge
pyasn1-modules            0.2.7                      py_0    conda-forge
pybigwig                  0.3.18           py39h015b436_1    bioconda
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pydeprecate               0.3.1              pyhd8ed1ab_0    conda-forge
pyfaidx                   0.6.3.1            pyh5e36f6f_0    bioconda
pyfasta                   0.5.2                      py_1    bioconda
pyjwt                     2.3.0              pyhd8ed1ab_1    conda-forge
pyopenssl                 21.0.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.0.4              pyhd3eb1b0_0    defaults
pyperclip                 1.8.2                    pypi_0    pypi
pyqt                      5.9.2            py39h2531618_6    defaults
pyranges                  0.0.113            pyh5e36f6f_0    bioconda
pyrle                     0.0.33           py39h38f01e4_0    bioconda
pysocks                   1.7.1            py39hf3d152e_4    conda-forge
python                    3.9.7                h12debd9_1    defaults
python-dateutil           2.8.2              pyhd3eb1b0_0    defaults
python-slugify            5.0.2                    pypi_0    pypi
python_abi                3.9                      2_cp39    conda-forge
pytorch                   1.10.0          py3.9_cuda11.1_cudnn8.0.5_0    pytorch
pytorch-lightning         1.5.7              pyhd8ed1ab_0    conda-forge
pytorch-mutex             1.0                        cuda    pytorch
pytz                      2021.3             pyhd3eb1b0_0    defaults
pyu2f                     0.1.5              pyhd8ed1ab_0    conda-forge
pyyaml                    6.0              py39h7f8727e_1    defaults
qt                        5.9.7                h5867ecd_1    defaults
readline                  8.1                  h27cfd23_0    defaults
reformer-pytorch          1.4.4                    pypi_0    pypi
regex                     2022.7.9                 pypi_0    pypi
related                   0.7.2                    pypi_0    pypi
requests                  2.26.0             pyhd8ed1ab_1    conda-forge
requests-oauthlib         1.3.0              pyh9f0ad1d_0    conda-forge
responses                 0.18.0                   pypi_0    pypi
routing-transformer       1.6.1                    pypi_0    pypi
rsa                       4.8                pyhd8ed1ab_0    conda-forge
scikit-learn              1.0.1            py39h51133e4_0    defaults
scipy                     1.6.2            py39had2a1c9_1    defaults
seaborn                   0.11.2             pyhd3eb1b0_0    defaults
setuptools                58.0.4           py39h06a4308_0    defaults
simplejson                3.17.6                   pypi_0    pypi
sinkhorn-transformer      0.11.4                   pypi_0    pypi
sip                       4.19.13          py39h2531618_0    defaults
six                       1.16.0             pyhd3eb1b0_0    defaults
sorted_nearest            0.0.33           py39hcbe4a3b_0    bioconda
sqlalchemy                1.4.35                   pypi_0    pypi
sqlite                    3.36.0               hc218d9a_0    defaults
stevedore                 3.5.0                    pypi_0    pypi
tabulate                  0.8.9            py39h06a4308_0    defaults
tensorboard               2.7.0              pyhd8ed1ab_0    conda-forge
tensorboard-data-server   0.6.0            py39h95dcef6_1    conda-forge
tensorboard-plugin-wit    1.8.0              pyh44b312d_0    conda-forge
text-unidecode            1.3                      pypi_0    pypi
thop                      0.0.31-2005241907          pypi_0    pypi
threadpoolctl             2.2.0              pyh0d69192_0    defaults
tinydb                    4.6.1                    pypi_0    pypi
tk                        8.6.11               h1ccaba5_0    defaults
tokenizers                0.12.1                   pypi_0    pypi
torchaudio                0.10.0               py39_cu111    pytorch
torchmetrics              0.6.2              pyhd8ed1ab_0    conda-forge
torchvision               0.11.1               py39_cu111    pytorch
tornado                   6.1              py39h27cfd23_0    defaults
tqdm                      4.62.3             pyhd8ed1ab_0    conda-forge
transformers              4.20.1                   pypi_0    pypi
triton                    1.0.0                    pypi_0    pypi
typing-extensions         4.3.0                    pypi_0    pypi
tzdata                    2021e                hda174b7_0    defaults
urllib3                   1.26.7             pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.5                    pypi_0    pypi
werkzeug                  2.0.1              pyhd8ed1ab_0    conda-forge
wheel                     0.37.0             pyhd3eb1b0_1    defaults
xxhash                    3.0.0                    pypi_0    pypi
xz                        5.2.5                h7b6447c_0    defaults
yaml                      0.2.5                h7b6447c_0    anaconda
yarl                      1.7.2            py39h3811e60_1    conda-forge
zarr                      2.11.3                   pypi_0    pypi
zipp                      3.6.0              pyhd8ed1ab_0    conda-forge
zlib                      1.2.11               h7b6447c_3    defaults
zstd                      1.4.9                haebb681_0    defaults