search

TileDB-Inc / tiledbsoma-feedstock

A conda-smithy repository for tiledbsoma.
BSD 3-Clause "New" or "Revised" License
3 stars 3 forks source link

Test out unbreak of R build #127

Closed johnkerl closed 5 months ago

johnkerl commented 5 months ago

Context:

johnkerl commented 5 months ago

@jdblischak at https://dev.azure.com/TileDB-Inc/CI/_build/results?buildId=38877&view=logs&j=58ac6aab-c4bc-5de2-2894-98e408cc8ec9&t=933f325c-924e-533d-4d95-e93b5843ce8b

- package libtiledbsoma-1.9.5rc0-haede419_0 requires tiledb >=2.21.2,<2.22.0a0, but none of the providers can be installed

-- I think I need a tiledb/for-cloud feedstock PR perhaps ... 👀

johnkerl commented 5 months ago

Derp, nope ... https://github.com/TileDB-Inc/tiledb-feedstock-for-cloud/pull/22 🤔

johnkerl commented 5 months ago

@jdblischak additionally I'm confused by the solver error

└─ libtiledbsoma 1.9.5rc0 haede419_0 is not installable because it requires
   └─ tiledb >=2.21.2,<2.22.0a0 , which requires
      └─ aws-crt-cpp >=0.26.4,<0.26.5.0a0 , which conflicts with any installable versions previously reported.

since in the recipe/meta.yaml in this repo we have only

tiledb >=2.21.0,<2.22

🤔

johnkerl commented 5 months ago

Also puzzled this is mac-only, not Linux ... :think

johnkerl commented 5 months ago

At https://anaconda.org/conda-forge/tiledb/files I see for example

| [osx-64/tiledb-2.21.2-h0d80af6_0.conda](https://anaconda.org/conda-forge/tiledb/2.21.2/download/osx-64/tiledb-2.21.2-h0d80af6_0.conda)

🤔

johnkerl commented 5 months ago

@jdblischak @ihnorton where does 

└─ libtiledbsoma 1.9.5rc0 haede419_0 is not installable because it requires
   └─ tiledb >=2.21.2,<2.22.0a0 , which requires

come from?

jdblischak commented 5 months ago

tiledb 2.21.2 comes from the "run exports" of the tiledb recipe. Since libtiledbsoma was built against tiledb 2.21.2, it requires tiledb >=2.21.2,<2.22. This is the same problem we had last week with the cloud Docker files (which were pinning tiledb 2.21.1). For the cloud Docker files, we just had to update the TileDB env var pin to 2.21.2. In this case we'll need to migrate the aws version, so hopefully the solver error will be fixed by https://github.com/conda-forge/tiledb-feedstock/pull/271

jdblischak commented 5 months ago

I just merged https://github.com/conda-forge/tiledb-feedstock/pull/271. The build to monitor is at https://github.com/conda-forge/tiledb-feedstock/commit/95e102d1fc50d80e101a12bfbd86415edefc806c

johnkerl commented 5 months ago

@jdblischak OK thank you!

Meanwhile TileDB-SOMA -- as a package outside of Conda -- depends on 2.21.1.

I tried https://github.com/single-cell-data/TileDB-SOMA/pull/2424 but I found that TileDB-Py https://github.com/TileDB-Inc/TileDB-Py/releases/tag/0.27.1 is getting 2.21.1 and there is as yet no TileDB-Py 0.27.2 using 2.21.2. I'm not sure what's going to happen next but I'll continue working on this.

jdblischak commented 5 months ago

Meanwhile TileDB-SOMA -- as a package outside of Conda -- depends on 2.21.1.

If that is so, we would need to update all the pins in meta.yaml to reflect this stricter requirement

I found that TileDB-Py https://github.com/TileDB-Inc/TileDB-Py/releases/tag/0.27.1 is getting 2.21.1 and there is as yet no TileDB-Py 0.27.2 using 2.21.2

Have we confirmed that is a strict requirement? While tiledb-py 0.27.1 bundles tiledb 2.21.1 (eg in its wheels uploaded to PyPI), it's my understanding that 0.27.1 should be compatible with any tiledb 2.21.*. That is how the tiledb-py-feedstock operates, so if my assumption is untrue, we should also fix that recipe

https://github.com/conda-forge/tiledb-py-feedstock/blob/0656ff28f3f855c63d993790c38c1733bfa1e55e/recipe/meta.yaml#L35

As further evidence, the only difference between tiledb-py 0.27.0 and 0.27.1 is the bundled version of tiledb, which increases my confidence in my assertion that the 0.27.1 release was purely a mechanism for deploying tiledb 2.21.1 via the PyPI wheels

https://github.com/TileDB-Inc/TileDB-Py/compare/0.27.0...0.27.1

ihnorton commented 5 months ago

it's my understanding that 0.27.1 should be compatible with any tiledb 2.21.*

Correct

johnkerl commented 5 months ago

I found that TileDB-Py https://github.com/TileDB-Inc/TileDB-Py/releases/tag/0.27.1 is getting 2.21.1 and there is as yet no TileDB-Py 0.27.2 using 2.21.2

Have we confirmed that is a strict requirement? While tiledb-py 0.27.1 bundles tiledb 2.21.1 (eg in its wheels uploaded to PyPI), it's my understanding that 0.27.1 should be compatible with any tiledb 2.21.*. That is how the tiledb-py-feedstock operates, so if my assumption is untrue, we should also fix that recipe

@jdblischak your theoretical reasoning is 100% solid & I wish it were that way. However on https://github.com/single-cell-data/TileDB-SOMA/pull/2424#issuecomment-2045916104 I found that 2.21.1 was actually getting found (admittedly, outside of Conda).

We need to thread today's needle with regard to:

and see if we can get green CI on this feedstock PR. It's not clear to me yet how this will all unfold.

jdblischak commented 5 months ago

I found that 2.21.1 was actually getting found (admittedly, outside of Conda).

I don't think that should affect the conda builds. That upstream CI job installed tiledb-py from a PyPI wheel, which bundles its own copy of libtiledb. I agree this is a general problem with non-conda builds, and something I continuously struggle with in my various nightly builds (ie the nightly builds only make sense if all the clients are using the same underlying version of the nightly libtiledb). But with the various conda builds, they will all be linked to the conda-installed libtiledb

Using cached tiledb-0.27.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.6 kB)
johnkerl commented 5 months ago

This seems to be superseded by #129 so I'll abandon this attempt.

johnkerl commented 5 months ago

https://github.com/single-cell-data/TileDB-SOMA/issues/2414

[sc-44854]