search

jrzaurin / pytorch-widedeep

A flexible package for multimodal-deep-learning to combine tabular data with text and images using Wide and Deep models in Pytorch
Apache License 2.0
1.3k stars 190 forks source link

OSError when importing the package #193

Closed GreyGuoweiChen closed 1 year ago

GreyGuoweiChen commented 1 year ago

Traceback (most recent call last): File "/Users/graveschan/PycharmProjects/virus_host/WD_model.py", line 1, in from pytorch_widedeep import Trainer File "/Users/graveschan/software/pytorch-widedeep/pytorch_widedeep/init.py", line 17, in from pytorch_widedeep.training import Trainer, BayesianTrainer File "/Users/graveschan/software/pytorch-widedeep/pytorch_widedeep/training/init.py", line 1, in from pytorch_widedeep.training.trainer import Trainer File "/Users/graveschan/software/pytorch-widedeep/pytorch_widedeep/training/trainer.py", line 11, in from torchmetrics import Metric as TorchMetric File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchmetrics/init.py", line 14, in from torchmetrics import functional # noqa: E402 File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchmetrics/functional/init.py", line 14, in from torchmetrics.functional.audio._deprecated import _permutation_invariant_training as permutation_invariant_training File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchmetrics/functional/audio/init.py", line 14, in from torchmetrics.functional.audio.pit import permutation_invariant_training, pit_permutate File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchmetrics/functional/audio/pit.py", line 22, in from torchmetrics.utilities import rank_zero_warn File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchmetrics/utilities/init.py", line 14, in from torchmetrics.utilities.checks import check_forward_full_state_property File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchmetrics/utilities/checks.py", line 25, in from torchmetrics.metric import Metric File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchmetrics/metric.py", line 30, in from torchmetrics.utilities.data import ( File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchmetrics/utilities/data.py", line 22, in from torchmetrics.utilities.imports import _TORCH_GREATER_EQUAL_1_12, _XLA_AVAILABLE File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchmetrics/utilities/imports.py", line 50, in _TORCHAUDIO_GREATER_EQUAL_0_10: Optional[bool] = compare_version("torchaudio", operator.ge, "0.10.0") File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/lightning_utilities/core/imports.py", line 73, in compare_version pkg = importlib.import_module(package) File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchaudio/init.py", line 1, in from torchaudio import ( # noqa: F401 File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchaudio/_extension.py", line 135, in _init_extension() File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchaudio/_extension.py", line 105, in _init_extension _load_lib("libtorchaudio") File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchaudio/_extension.py", line 52, in _load_lib torch.ops.load_library(path) File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torch/_ops.py", line 573, in load_library ctypes.CDLL(path) File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/ctypes/init.py", line 373, in init self._handle = _dlopen(self._name, mode) OSError: dlopen(/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchaudio/lib/libtorchaudio.so, 0x0006): Symbol not found: __ZN2at8internal15invoke_parallelExxxRKNSt3__18functionIFvxxEEE Referenced from: /Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchaudio/lib/libtorchaudio.so Expected in: /Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torch/lib/libtorch_cpu.dylib

This problem happens to me, when importing the package

from pytorch_widedeep import Trainer

The environment is MacOS M1 arm, miniforge, and packages are as:

pytorch 1.13.1 cpu_py38h0b410dd_1 conda-forge pytorch-widedeep 1.3.2 dev_0 torchaudio 0.13.1 py38_cpu pytorch torchmetrics 1.2.0 pypi_0 pypi torchvision 0.14.1 cpu_py38h4eaac98_1 conda-forge

jrzaurin commented 1 year ago

Hey @GreyGuoweiChen let us look into it. I am traveling these week, but I will do my best :)

Also the error seems to be related to the interplay between torchmetrics and torchaudio 🤔? have you tried without installing torchaudio. Even if you need that library it would be good to see if the issue raises from that library.

maybe @gradientsky has seen this?

GreyGuoweiChen commented 1 year ago

Hello @jrzaurin ,

Thanks for your timely reply, and wish you a happy travel. I have some tries and have avoided this bug.

Firstly, I tried to upgrade the pytorch package to 2.0.1 sa below. 

pytorch                   2.0.1                   py3.8_0    pytorch
pytorch-widedeep          1.3.2                     dev_0    <develop>
torchaudio                2.0.2                  py38_cpu    pytorch
torchmetrics              1.2.0                    pypi_0    pypi
torchvision               0.15.2                 py38_cpu    pytorch

And the error turns to

Traceback (most recent call last): File "/Users/graveschan/PycharmProjects/virus_host/WD_model.py", line 1, in from pytorch_widedeep import Trainer File "/Users/graveschan/software/pytorch-widedeep/pytorch_widedeep/init.py", line 9, in from pytorch_widedeep.utils import ( File "/Users/graveschan/software/pytorch-widedeep/pytorch_widedeep/utils/init.py", line 1, in from pytorch_widedeep.utils.text_utils import ( File "/Users/graveschan/software/pytorch-widedeep/pytorch_widedeep/utils/text_utils.py", line 7, in from pytorch_widedeep.utils.fastai_transforms import Vocab, Tokenizer File "/Users/graveschan/software/pytorch-widedeep/pytorch_widedeep/utils/fastai_transforms.py", line 21, in from pytorch_widedeep.wdtypes import ( File "/Users/graveschan/software/pytorch-widedeep/pytorch_widedeep/wdtypes.py", line 23, in from torchvision.transforms import ( File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchvision/init.py", line 6, in from torchvision import datasets, io, models, ops, transforms, utils File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchvision/datasets/init.py", line 1, in from ._optical_flow import FlyingChairs, FlyingThings3D, HD1K, KittiFlow, Sintel File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/torchvision/datasets/_optical_flow.py", line 10, in from PIL import Image File "/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/PIL/Image.py", line 103, in from . import _imaging as core ImportError: dlopen(/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/PIL/_imaging.cpython-38-darwin.so, 0x0002): Library not loaded: @rpath/libtiff.5.dylib Referenced from: /Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/PIL/_imaging.cpython-38-darwin.so Reason: tried: '/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/PIL/../../../libtiff.5.dylib' (no such file), '/Users/graveschan/miniforge3/envs/host_pred/lib/python3.8/site-packages/PIL/../../../libtiff.5.dylib' (no such file), '/Users/graveschan/miniforge3/envs/host_pred/bin/../lib/libtiff.5.dylib' (no such file), '/Users/graveschan/miniforge3/envs/host_pred/bin/../lib/libtiff.5.dylib' (no such file), '/usr/local/lib/libtiff.5.dylib' (no such file), '/usr/lib/libtiff.5.dylib' (no such file)

And then I downgraded the Pillow to 9.3.0. This solves the error as above.

jrzaurin commented 1 year ago

Hey @GreyGuoweiChen many thanks for all this info. I plan to review dependencies also related to another issue as soon as I can and this will be helpful!

thanks again and thanks for opening the issue :)

jrzaurin commented 1 year ago

requirements have been updated and unit tests passed in GPU and CPU, with python versions 3.8, 3.9 and 3.10.

Hopefully there will be no dependencies issues directly related to the library