search

PonteIneptique / YALTAi

You Actually Look Twice At it
GNU General Public License v3.0
29 stars 1 forks source link

dependency conflicts #16

Closed gabays closed 1 month ago

gabays commented 11 months ago

I am trying to use YALTAi with colab. Code is here:

https://github.com/gabays/CHR_2023/blob/main/CHR_digital_diplomacy.ipynb

When doing pip install YALTAi I get the following error:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
lida 0.0.10 requires fastapi, which is not installed.
lida 0.0.10 requires kaleido, which is not installed.
lida 0.0.10 requires python-multipart, which is not installed.
lida 0.0.10 requires uvicorn, which is not installed.
bigframes 0.15.0 requires tabulate>=0.9, but you have tabulate 0.8.10 which is incompatible.
torchaudio 2.1.0+cu118 requires torch==2.1.0, but you have torch 2.0.1 which is incompatible.
torchdata 0.7.0 requires torch==2.1.0, but you have torch 2.0.1 which is incompatible.
torchtext 0.16.0 requires torch==2.1.0, but you have torch 2.0.1 which is incompatible.

At the beginning it works, but when executing it seems to create problems

PonteIneptique commented 11 months ago

Which version are you trying to install? What's the error you're getting at execution time?

On Sat, 2 Dec 2023, 7:42 pm Simon Gabay, @.***> wrote:

I am trying to use YALTAi with colab. Code is here:

https://github.com/gabays/CHR_2023/blob/main/CHR_digital_diplomacy.ipynb

When doing pip install YALTAi I get the following error:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. lida 0.0.10 requires fastapi, which is not installed. lida 0.0.10 requires kaleido, which is not installed. lida 0.0.10 requires python-multipart, which is not installed. lida 0.0.10 requires uvicorn, which is not installed. bigframes 0.15.0 requires tabulate>=0.9, but you have tabulate 0.8.10 which is incompatible. torchaudio 2.1.0+cu118 requires torch==2.1.0, but you have torch 2.0.1 which is incompatible. torchdata 0.7.0 requires torch==2.1.0, but you have torch 2.0.1 which is incompatible. torchtext 0.16.0 requires torch==2.1.0, but you have torch 2.0.1 which is incompatible.

At the beginning it works, but when executing it seems to create problems

— Reply to this email directly, view it on GitHub https://github.com/PonteIneptique/YALTAi/issues/16, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOXEZQUERQTFF2PC2NZ4SLYHNZBZAVCNFSM6AAAAABAEF6F4WVHI2DSMVQWIX3LMV43ASLTON2WKOZSGAZDEMJWGAYTSMY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

gabays commented 11 months ago

YALTAi that comes with pip. I will check.

Here is the traceback

Traceback (most recent call last):
  File "/usr/local/bin/yaltai", line 5, in <module>
    from yaltai.yaltai import cli
  File "/usr/local/lib/python3.10/dist-packages/yaltai/yaltai.py", line 20, in <module>
    from kraken.kraken import message
  File "/usr/local/lib/python3.10/dist-packages/kraken/kraken.py", line 35, in <module>
    from kraken.lib.progress import KrakenProgressBar, KrakenDownloadProgressBar
  File "/usr/local/lib/python3.10/dist-packages/kraken/lib/progress.py", line 22, in <module>
    import pytorch_lightning as pl
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/__init__.py", line 26, in <module>
    from pytorch_lightning.callbacks import Callback  # noqa: E402
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/callbacks/__init__.py", line 14, in <module>
    from pytorch_lightning.callbacks.batch_size_finder import BatchSizeFinder
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/callbacks/batch_size_finder.py", line 24, in <module>
    from pytorch_lightning.callbacks.callback import Callback
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/callbacks/callback.py", line 22, in <module>
    from pytorch_lightning.utilities.types import STEP_OUTPUT
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/utilities/types.py", line 40, in <module>
    from torchmetrics import Metric
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/__init__.py", line 22, in <module>
    from torchmetrics import functional  # noqa: E402
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/functional/__init__.py", line 14, in <module>
    from torchmetrics.functional.audio._deprecated import _permutation_invariant_training as permutation_invariant_training
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/functional/audio/__init__.py", line 14, in <module>
    from torchmetrics.functional.audio.pit import permutation_invariant_training, pit_permutate
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/functional/audio/pit.py", line 22, in <module>
    from torchmetrics.utilities import rank_zero_warn
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/utilities/__init__.py", line 14, in <module>
    from torchmetrics.utilities.checks import check_forward_full_state_property
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/utilities/checks.py", line 25, in <module>
    from torchmetrics.metric import Metric
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/metric.py", line 30, in <module>
    from torchmetrics.utilities.data import (
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/utilities/data.py", line 22, in <module>
    from torchmetrics.utilities.imports import _TORCH_GREATER_EQUAL_1_12, _XLA_AVAILABLE
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/utilities/imports.py", line 54, in <module>
    _TORCHAUDIO_GREATER_EQUAL_0_10: Optional[bool] = compare_version("torchaudio", operator.ge, "0.10.0")
  File "/usr/local/lib/python3.10/dist-packages/lightning_utilities/core/imports.py", line 77, in compare_version
    pkg = importlib.import_module(package)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/local/lib/python3.10/dist-packages/torchaudio/__init__.py", line 1, in <module>
    from . import (  # noqa: F401
  File "/usr/local/lib/python3.10/dist-packages/torchaudio/_extension/__init__.py", line 45, in <module>
    _load_lib("libtorchaudio")
  File "/usr/local/lib/python3.10/dist-packages/torchaudio/_extension/utils.py", line 64, in _load_lib
    torch.ops.load_library(path)
  File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 643, in load_library
    ctypes.CDLL(path)
  File "/usr/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /usr/local/lib/python3.10/dist-packages/torchaudio/lib/libtorchaudio.so: undefined symbol: _ZN3c104cuda9SetDeviceEi
gabays commented 11 months ago

PyPI says I am supposed to use 1.0.0 since it distributes this version:

https://pypi.org/project/YALTAi/1.0.0/

gabays commented 11 months ago

Solution for dependencies conflict: install kraken and YALTAi at once:

!pip install YALTAi

Traceback remains though

Traceback (most recent call last):
  File "/usr/local/bin/yaltai", line 5, in <module>
    from yaltai.yaltai import cli
  File "/usr/local/lib/python3.10/dist-packages/yaltai/yaltai.py", line 20, in <module>
    from kraken.kraken import message
  File "/usr/local/lib/python3.10/dist-packages/kraken/kraken.py", line 35, in <module>
    from kraken.lib.progress import KrakenProgressBar, KrakenDownloadProgressBar
  File "/usr/local/lib/python3.10/dist-packages/kraken/lib/progress.py", line 22, in <module>
    import pytorch_lightning as pl
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/__init__.py", line 26, in <module>
    from pytorch_lightning.callbacks import Callback  # noqa: E402
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/callbacks/__init__.py", line 14, in <module>
    from pytorch_lightning.callbacks.batch_size_finder import BatchSizeFinder
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/callbacks/batch_size_finder.py", line 24, in <module>
    from pytorch_lightning.callbacks.callback import Callback
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/callbacks/callback.py", line 22, in <module>
    from pytorch_lightning.utilities.types import STEP_OUTPUT
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/utilities/types.py", line 40, in <module>
    from torchmetrics import Metric
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/__init__.py", line 22, in <module>
    from torchmetrics import functional  # noqa: E402
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/functional/__init__.py", line 14, in <module>
    from torchmetrics.functional.audio._deprecated import _permutation_invariant_training as permutation_invariant_training
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/functional/audio/__init__.py", line 14, in <module>
    from torchmetrics.functional.audio.pit import permutation_invariant_training, pit_permutate
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/functional/audio/pit.py", line 22, in <module>
    from torchmetrics.utilities import rank_zero_warn
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/utilities/__init__.py", line 14, in <module>
    from torchmetrics.utilities.checks import check_forward_full_state_property
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/utilities/checks.py", line 25, in <module>
    from torchmetrics.metric import Metric
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/metric.py", line 30, in <module>
    from torchmetrics.utilities.data import (
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/utilities/data.py", line 22, in <module>
    from torchmetrics.utilities.imports import _TORCH_GREATER_EQUAL_1_12, _XLA_AVAILABLE
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/utilities/imports.py", line 54, in <module>
    _TORCHAUDIO_GREATER_EQUAL_0_10: Optional[bool] = compare_version("torchaudio", operator.ge, "0.10.0")
  File "/usr/local/lib/python3.10/dist-packages/lightning_utilities/core/imports.py", line 77, in compare_version
    pkg = importlib.import_module(package)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/local/lib/python3.10/dist-packages/torchaudio/__init__.py", line 1, in <module>
    from . import (  # noqa: F401
  File "/usr/local/lib/python3.10/dist-packages/torchaudio/_extension/__init__.py", line 45, in <module>
    _load_lib("libtorchaudio")
  File "/usr/local/lib/python3.10/dist-packages/torchaudio/_extension/utils.py", line 64, in _load_lib
    torch.ops.load_library(path)
  File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 643, in load_library
    ctypes.CDLL(path)
  File "/usr/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /usr/local/lib/python3.10/dist-packages/torchaudio/lib/libtorchaudio.so: undefined symbol: _ZN3c104cuda9SetDeviceEi
gabays commented 11 months ago

So with only

!pip install YALTAi

it first install Torch 2.0.1

Collecting torch~=2.0.1 (from kraken==4.3.13->YALTAi)
  Downloading torch-2.0.1-cp310-cp310-manylinux1_x86_64.whl (619.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 619.9/619.9 MB 1.8 MB/s eta 0:00:00

Then it uninstalls Torch 2.1.0

  Attempting uninstall: torch
    Found existing installation: torch 2.1.0+cu118
    Uninstalling torch-2.1.0+cu118:
      Successfully uninstalled torch-2.1.0+cu118

Then there is a warning with torchaudio et alii because I don't have 2.10 but 2.0.1

torchaudio 2.1.0+cu118 requires torch==2.1.0, but you have torch 2.0.1 which is incompatible.
torchdata 0.7.0 requires torch==2.1.0, but you have torch 2.0.1 which is incompatible.
torchtext 0.16.0 requires torch==2.1.0, but you have torch 2.0.1 which is incompatible.

And when I use YALTAi there is a bug with torchaudio

OSError: /usr/local/lib/python3.10/dist-packages/torchaudio/lib/libtorchaudio.so: undefined symbol: _ZN3c104cuda9SetDeviceEi
Svetlana-Yatsyk commented 11 months ago

I experience the same bug with torchaudio when installing Yaltai 1.0.0.

I run

!git clone https://github.com/PonteIneptique/YALTAi/
!pip install -r /content/YALTAi/requirements.txt
!pip install --no-build-isolation YALTAi

and the traceback is similar to what Simon posted above:

Traceback (most recent call last):
  File "/usr/local/bin/yaltai", line 5, in <module>
    from yaltai.yaltai import cli
  File "/usr/local/lib/python3.10/dist-packages/yaltai/yaltai.py", line 20, in <module>
    from kraken.kraken import message
  File "/usr/local/lib/python3.10/dist-packages/kraken/kraken.py", line 35, in <module>
    from kraken.lib.progress import KrakenProgressBar, KrakenDownloadProgressBar
  File "/usr/local/lib/python3.10/dist-packages/kraken/lib/progress.py", line 22, in <module>
    import pytorch_lightning as pl
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/__init__.py", line 26, in <module>
    from pytorch_lightning.callbacks import Callback  # noqa: E402
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/callbacks/__init__.py", line 14, in <module>
    from pytorch_lightning.callbacks.batch_size_finder import BatchSizeFinder
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/callbacks/batch_size_finder.py", line 24, in <module>
    from pytorch_lightning.callbacks.callback import Callback
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/callbacks/callback.py", line 22, in <module>
    from pytorch_lightning.utilities.types import STEP_OUTPUT
  File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/utilities/types.py", line 40, in <module>
    from torchmetrics import Metric
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/__init__.py", line 22, in <module>
    from torchmetrics import functional  # noqa: E402
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/functional/__init__.py", line 14, in <module>
    from torchmetrics.functional.audio._deprecated import _permutation_invariant_training as permutation_invariant_training
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/functional/audio/__init__.py", line 14, in <module>
    from torchmetrics.functional.audio.pit import permutation_invariant_training, pit_permutate
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/functional/audio/pit.py", line 22, in <module>
    from torchmetrics.utilities import rank_zero_warn
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/utilities/__init__.py", line 14, in <module>
    from torchmetrics.utilities.checks import check_forward_full_state_property
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/utilities/checks.py", line 25, in <module>
    from torchmetrics.metric import Metric
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/metric.py", line 30, in <module>
    from torchmetrics.utilities.data import (
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/utilities/data.py", line 22, in <module>
    from torchmetrics.utilities.imports import _TORCH_GREATER_EQUAL_1_12, _XLA_AVAILABLE
  File "/usr/local/lib/python3.10/dist-packages/torchmetrics/utilities/imports.py", line 54, in <module>
    _TORCHAUDIO_GREATER_EQUAL_0_10: Optional[bool] = compare_version("torchaudio", operator.ge, "0.10.0")
  File "/usr/local/lib/python3.10/dist-packages/lightning_utilities/core/imports.py", line 77, in compare_version
    pkg = importlib.import_module(package)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/local/lib/python3.10/dist-packages/torchaudio/__init__.py", line 1, in <module>
    from . import (  # noqa: F401
  File "/usr/local/lib/python3.10/dist-packages/torchaudio/_extension/__init__.py", line 45, in <module>
    _load_lib("libtorchaudio")
  File "/usr/local/lib/python3.10/dist-packages/torchaudio/_extension/utils.py", line 64, in _load_lib
    torch.ops.load_library(path)
  File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 643, in load_library
    ctypes.CDLL(path)
  File "/usr/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /usr/local/lib/python3.10/dist-packages/torchaudio/lib/libtorchaudio.so: undefined symbol: _ZN3c104cuda9SetDeviceEi
gabays commented 11 months ago

Hello Svetlana: are you on colab? If yes, do that:

!pip install YALTAi
!pip uninstall torchaudio torch torchvision torchtext torchdata -y
!pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 torchtext==0.15.2 torchdata==0.6.1 --index-url https://download.pytorch.org/whl/cu117
Svetlana-Yatsyk commented 11 months ago

UPD2: I fail to load the segmented images to eScriptorium though, getting Import failed because 'get() returned more than one BlockType -- it returned 2!' error.

UPD: never mind, I fixed the dependency conflict and used a model compatible with Yaltai 1.0.0 (and not an old one) and it worked

Hello, thank you for the suggestion ! I did that (following your collab), and Yaltai seems to be installed, however, when I try to segment some images with it I get 

scikit-learn version 1.2.2 is not supported. Minimum required version: 0.17. Maximum required version: 1.1.2. Disabling scikit-learn conversion API.
XGBoost version 2.0.2 has not been tested with coremltools. You may run into unexpected errors. XGBoost 1.4.2 is the most recent version that has been tested.
2023-12-12 21:08:48.475543: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2023-12-12 21:08:48.475613: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2023-12-12 21:08:48.475649: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2023-12-12 21:08:49.816035: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
TensorFlow version 2.14.0 has not been tested with coremltools. You may run into unexpected errors. TensorFlow 2.12.0 is the most recent version that has been tested.
Torch version 2.0.1+cu117 has not been tested with coremltools. You may run into unexpected errors. Torch 2.0.0 is the most recent version that has been tested.
Loading ANN /content/gdrive/MyDrive/yaltai/segm_baselines_medieval_Thibault.mlmodel Segmenting  ✗
[12/12/23 21:08:52] ERROR    Failed processing
gabays commented 11 months ago

Can you share your notebook?

Svetlana-Yatsyk commented 11 months ago

I resolved the import problem by deleting all the zone types present in the document's ontology on eScriptorium.

However, I would greatly appreciate it if you could take a look at this Colab where I attempt to install Yaltai v0.0.1rc4 and explain why I am unsuccessful.

gabays commented 11 months ago

Hello Svetlana, I had a quick look, but I don't find the error you mentioned above