constantinpape / z5

Lightweight C++ and Python interface for datasets in zarr and N5 format
MIT License
111 stars 27 forks source link

Add Python 3.11 to CI builds #212

Closed DimitriPapadopoulos closed 1 year ago

joshmoore commented 2 years ago

Looks like there are quite a number of incompatibilities in the stack:

  UnsatisfiableError: The following specifications were found to be incompatible with each other:

  Output in format: Requested package -> Available versions

  Package xz conflicts for:
  boost-cpp[version='>=1.63'] -> zstd[version='>=1.5.2,<1.6.0a0'] -> xz[version='>=5.2.4,<6.0a0|>=5.2.5,<6.0a0']
  cmake -> xz[version='5.0.*|5.2.*|>=5.2.3,<5.3.0a0|>=5.2.4,<5.3.0a0|>=5.2.5,<5.3.0a0|>=5.2.6,<5.3.0a0|>=5.2.5,<6.0a0|>=5.2.4,<6.0a0|>=5.2.3,<6.0a0']

etc. etc. etc.
jakirkham commented 2 years ago

Interesting. In conda-build 2 we use to mix build tools and dependencies in the same environment. This caused a variety of headaches, which finally led to us splitting them in conda-build 3 into build tools and the host environment (where libraries are linked to).

The resolution error here seems to imply that build tools (like cmake) are installed in the same environment as library dependencies that z5 links to. A reasonable fix would be to split these environments apart. This is a bit easier to do when building a recipe (since conda-build has this logic baked in), but one might be able to achieve the same thing just with conda (without conda-build) by using environment stacking. IOW create an environment of build tools and another environment with libraries z5 links to and then stack them.

constantinpape commented 2 years ago

Yes, indeed there is just a single env for now: https://github.com/constantinpape/z5/blob/master/.github/workflows/environment.yaml.

This is a bit easier to do when building a recipe (since conda-build has this logic baked in), but one might be able to achieve the same thing just with conda (without conda-build) by using environment stacking. IOW create an environment of build tools and another environment with libraries z5 links to and then stack them.

Interesting. I don't have time to look into this myself right now, but happy to review / help out a bit.

jakirkham commented 2 years ago

Nor do I unfortunately. Was hoping the info would be useful/helpful for whoever might have interest & time

DimitriPapadopoulos commented 1 year ago

It works without a single change on my side. Incompatibilities have been fixed directly in the stack.