search

ome / ome-zarr-py

Implementation of next-generation file format (NGFF) specifications for storing bioimaging data in the cloud.
https://pypi.org/project/ome-zarr
Other
146 stars 52 forks source link

Incompatibility with Poetry? #353

Open akhanf opened 4 months ago

akhanf commented 4 months ago

Hello,

The dependency solver seems to go on an infinite loop when trying to add ome-zarr to a poetry project.

Can be reproduced even with just ome-zarr as the only dependency:

alik@AFI-CBS-H-15:/local/scratch/test_ome_zarr$ poetry --version
Poetry (version 1.4.2)
alik@AFI-CBS-H-15:/local/scratch/test_ome_zarr$ python3.11 --version
Python 3.11.7
alik@AFI-CBS-H-15:/local/scratch/test_ome_zarr$ poetry init

This command will guide you through creating your pyproject.toml config.

Package name [test_ome_zarr]:  
Version [0.1.0]:  
Description []:  
Author [Ali Khan <alik@robarts.ca>, n to skip]:  
License []:  
Compatible Python versions [^3.8]:  

Would you like to define your main dependencies interactively? (yes/no) [yes] 
You can specify a package in the following forms:
  - A single name (requests): this will search for matches on PyPI
  - A name and a constraint (requests@^2.23.0)
  - A git url (git+https://github.com/python-poetry/poetry.git)
  - A git url with a revision (git+https://github.com/python-poetry/poetry.git#develop)
  - A file path (../my-package/my-package.whl)
  - A directory (../my-package/)
  - A url (https://example.com/packages/my-package-0.1.0.tar.gz)

Package to add or search for (leave blank to skip): ome-zarr
Found 20 packages matching ome-zarr
Showing the first 10 matches

Enter package # to add, or the complete package name if it is not listed []:
 [ 0] ome-zarr
 [ 1] napari-ome-zarr
 [ 2] bioio-ome-zarr
 [ 3] omero-web-zarr
 [ 4] mcd2zarr
 [ 5] iohub
 [ 6] ngff-zarr
 [ 7] omero-cli-zarr
 [ 8] ez-zarr
 [ 9] raw2tmap
 [ 10] 
 > 0
Enter the version constraint to require (or leave blank to use the latest version): 
Using version ^0.8.3 for ome-zarr

Add a package (leave blank to skip): 

Would you like to define your development dependencies interactively? (yes/no) [yes] no 
Generated file

[tool.poetry]
name = "test-ome-zarr"
version = "0.1.0"
description = ""
authors = ["Ali Khan <alik@robarts.ca>"]
readme = "README.md"
packages = [{include = "test_ome_zarr"}]

[tool.poetry.dependencies]
python = "^3.8"
ome-zarr = "^0.8.3"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Do you confirm generation? (yes/no) [yes] 
alik@AFI-CBS-H-15:/local/scratch/test_ome_zarr$ poetry install
Creating virtualenv test-ome-zarr--TtX-DE5-py3.8 in /localscratch/pypoetry_venv
Updating dependencies
Resolving dependencies... Downloading https://files.pythonhosted.org/packages/33/47/fc483df0b7ddeee987b6ff146c879d3556fcea82cc9aa420Resolving dependencies... Downloading https://files.pythonhosted.org/packages/33/47/fc483df0b7ddeee987b6ff146c879d3556fcea82cc9aa420Resolving dependencies... Downloading https://files.pythonhosted.org/packages/33/47/fc483df0b7ddeee987b6ff146c879d3556fcea82cc9aa420Resolving dependencies... Downloading https://files.pythonhosted.org/packages/33/47/fc483df0b7ddeee987b6ff146c879d3556fcea82cc9aa420Resolving dependencies... Downloading https://files.pythonhosted.org/packages/33/47/fc483df0b7ddeee987b6ff146c879d3556fcea82cc9aa420Resolving dependencies... Downloading https://files.pythonhosted.org/packages/33/47/fc483df0b7ddeee987b6ff146c879d3556fcea82cc9aa420Resolving dependencies... Downloading https://files.pythonhosted.org/packages/34/11/055802bf85abbb61988e6313e8b0a85167ee0795fc2c6141Resolving dependencies... Downloading https://files.pythonhosted.org/packages/34/11/055802bf85abbb61988e6313e8b0a85167ee0795fc2c6141Resolving dependencies... Downloading https://files.pythonhosted.org/packages/34/11/055802bf85abbb61988e6313e8b0a85167ee0795fc2c6141Resolving dependencies... Downloading https://files.pythonhosted.org/packages/34/11/055802bf85abbb61988e6313e8b0a85167ee0795fc2c6141Resolving dependencies... Downloading https://files.pythonhosted.org/packages/34/11/055802bf85abbb61988e6313e8b0a85167ee0795fc2c6141Resolving dependencies... Downloading https://files.pythonhosted.org/packages/be/f8/034752c5131c46e10364e4db241974f2eb6bb31bbfc43353Resolving dependencies... Downloading https://files.pythonhosted.org/packages/be/f8/034752c5131c46e10364e4db241974f2eb6bb31bbfc43353Resolving dependencies... Downloading https://files.pythonhosted.org/packages/be/f8/034752c5131c46e10364e4db241974f2eb6bb31bbfc43353Resolving dependencies... Downloading https://files.pythonhosted.org/packages/be/f8/034752c5131c46e10364e4db241974f2eb6bb31bbfc43353Resolving dependencies... Downloading https://files.pythonhosted.org/packages/be/f8/034752c5131c46e10364e4db241974f2eb6bb31bbfc43353Resolving dependencies... Downloading https://files.pythonhosted.org/packages/be/f8/034752c5131c46e10364e4db241974f2eb6bb31bbfc43353Resolving dependencies... Downloading https://files.pythonhosted.org/packages/d5/50/8aedb5ff1460e7c8527af15c6326115009e7c270ec705487Resolving dependencies... Downloading https://files.pythonhosted.org/packages/d5/50/8aedb5ff1460e7c8527af15c6326115009e7c270ec705487Resolving dependencies... Downloading https://files.pythonhosted.org/packages/d5/50/8aedb5ff1460e7c8527af15c6326115009e7c270ec705487Resolving dependencies... Downloading https://files.pythonhosted.org/packages/d5/50/8aedb5ff1460e7c8527af15c6326115009e7c270ec705487Resolving dependencies... Downloading https://files.pythonhosted.org/packages/d5/50/8aedb5ff1460e7c8527af15c6326115009e7c270ec705487Resolving dependencies... Downloading https://files.pythonhosted.org/packages/d5/50/8aedb5ff1460e7c8527af15c6326115009e7c270ec705487Resolving dependencies... Downloading https://files.pythonhosted.org/packages/fa/07/c6980120967a9fc76138eddd583d6ac47dd072922d6f66d7Resolving dependencies... Downloading https://files.pythonhosted.org/packages/fa/07/c6980120967a9fc76138eddd583d6ac47dd072922d6f66d7Resolving dependencies... Downloading https://files.pythonhosted.org/packages/fa/07/c6980120967a9fc76138eddd583d6ac47dd072922d6f66d7Resolving dependencies... Downloading https://files.pythonhosted.org/packages/fa/07/c6980120967a9fc76138eddd583d6ac47dd072922d6f66d7Resolving dependencies... Downloading https://files.pythonhosted.org/packages/fa/07/c6980120967a9fc76138eddd583d6ac47dd072922d6f66d7Resolving dependencies...

and so on.... I've let it go for a long time without success.. Maybe something in how the ome-zarr dependencies are pinned or described? I haven't dug into it in case it is obvious to anyone else.

Not urgent, as a simple workaround is to just install ome-zarr outside of poetry with just pip install ome-zarr which works fine and doesn't have any conflicts with other packages in my project.

Thanks! Ali

joshmoore commented 4 months ago

Thanks for the heads up, @akhanf! I don't know off-hand why that might be, but possibly related to the pinning of #342. Can you check if installing from the main branch helps?

akhanf commented 4 months ago

Had the same issue with main branch, but I made some headway here I think:

Looking at the verbose output when solving it looks like it was having an issue with boto3 -- I let it run for over an hour and it finally resolved dependencies, picking up boto3==1.7.84.

So now I can get it to solve quickly if I just pin boto3<1.8.0

However, that is a really old (Aug 2018) release of boto3, so probably not a great long-term solution.. Not sure what is conflicting to cause the problem.. Assuming it would have to be something pinned in the ome-zarr-py dependencies (or test dependencies)? I see there is an open PR for #346 that might resolve all this in any case..

joshmoore commented 4 months ago

@akhanf, I've updated #346. Can you give it a whirl?