search

TileDB-Inc / tiledbsoma-feedstock

A conda-smithy repository for tiledbsoma.
BSD 3-Clause "New" or "Revised" License
3 stars 3 forks source link

tiledbsoma 1.6.2 pre-check #74

Closed johnkerl closed 6 months ago

johnkerl commented 6 months ago

Following our established procedure: https://github.com/single-cell-data/TileDB-SOMA/wiki/Branches-and-releases

johnkerl commented 6 months ago

I do not understand https://github.com/TileDB-Inc/tiledbsoma-feedstock/pull/74/checks?check_run_id=20322245882 -- this attempted at a 1.6.2 is a bugfix-PR delta from 1.6.1 and I don't see why it won't solve

Specifically, the Azure error message is

package libtiledbsoma-1.6.2-hd763302_0 requires tiledb >=2.18.3,<2.19.0a0, but none of the providers can be installed

but I don't see why this does not solve given that 1.6.1 did solve (https://github.com/TileDB-Inc/tiledbsoma-feedstock/pull/68) ...

johnkerl commented 6 months ago

Additionally confusing: the bit requires tiledb >=2.18.3 given that this PR only has tiledb >=2.18.2,<2.19 ... 🤔

Additionally: the Azure raw log has

2024-01-09T22:53:43.5338780Z     tiledb:                   2.18.3-h4df5763_1           conda-forge

while https://anaconda.org/conda-forge/tiledb/2.18.3/download/osx-64/tiledb-2.18.3-h4df5763_1.conda does exist

jdblischak commented 6 months ago

Additionally confusing: the bit requires tiledb >=2.18.3 given that this PR only has tiledb >=2.18.2,<2.19

This is because of the "run exports" of the tiledb recipe. libtiledbsoma was built against libtiledb 2.18.3, so all the downstream clients have to install libtiledb 2.18.3 or greater as well

but I don't see why this does not solve given that 1.6.1 did solve (https://github.com/TileDB-Inc/tiledbsoma-feedstock/pull/68) ...

libtiledbsoma 1.6.1 was built against libtiledb 2.18.2

The conda solver issues have been brutal lately. These solver errors are the same problem that plagued the recent release of TileDB-VCF (https://github.com/TileDB-Inc/tiledb-vcf-feedstock/pull/105). There is some conflict between the libraries linked to libtiledb 2.18.3 and the libraries linked to something needed for the Python builds. I suspect libarrow/pyarrow, but I haven't been able to diagnose nor fix the solver issue despite trying multiple ideas last month

A short-term solution would be to pin to exactly tiledb 2.18.2, which should solve just like the 1.6.1 release did

johnkerl commented 6 months ago

@jdblischak thank you! https://github.com/TileDB-Inc/tiledbsoma-feedstock/pull/74/commits/ae9085e3dce7575384dafcf2108c94b045887b8a 🤞

jdblischak commented 6 months ago

The solver was failing because the tiledbsoma-py builds (except py37) and r-tiledbsoma builds require different versions of libtiledb:

Thus to enable flexibility in the patch version of libtiledb that is installed in the downstream clients, we had to do the following:

  1. Build libtiledbsoma against libtiledb 2.18.2. This results in the runtime requirement of tiledb >=2.18.2,<2.19.0a0, which is the behavior of max_pin="x.x" in the "run exports" of the libtiledb recipe
  2. Remove the strict run requirement of libtiledbsoma. This allows it to be co-installed with either libtiledb 2.18.2 or 2.18.3 as required by the downstream client

As I documented in https://github.com/conda-forge/tiledb-feedstock/pull/215, software built against a previous minor version of libtiledb (eg 2.18.2) is compatible at runtime with later patch releases (eg 2.18.3). The converse is not true. Software built against a later version (eg 2.18.3) is not guaranteed to be compatible at runtime with previous patch releases (eg 2.18.2). I learned this the hard way with the new API introduced in libtiledb 2.17.3

jdblischak commented 6 months ago

And a note to my future self or anyone else trying to fix conda solver issues in the future:

When I render the recipe locally, the tiledb 2.18.3 pin is inserted into the host requirement section of r-tiledbsoma. This is strange since tiledb is not a top-level requirement of r-tiledbsoma. The tiledbsoma-py recipe behaves as expected, ie only libtiledbsoma and tiledb-py are in the host requirements. If we can fix this behavior, this should make future tiledbsoma-feedstock updates easier

johnkerl commented 6 months ago

This pre-check PR has done its job following our established procedure: https://github.com/single-cell-data/TileDB-SOMA/wiki/Branches-and-releases