tiledbsoma 1.7.3 pre-check

[johnkerl]

Following our established procedure

See also https://github.com/single-cell-data/TileDB-SOMA/issues/2219 for tracking.

[jdblischak]

Finally making some progress. Now the R package can compile on osx-64.

However, now it is failing for the same reason as osx-arm64; it can't find libtiledbsoma.dylib. @nguyenv and I believe this is due to the changes introduced in https://github.com/single-cell-data/TileDB-SOMA/commit/b1de14e956fe926594639f79ffe2c2dd8ff1cf02. Would it be possible to drop this commit from the 1.7.3 backport? That should fix the osx-64 and osx-arm64 builds

[jdblischak]

Also, I've been so focused on the osx-64 build, I didn't notice that the linux-64 build is failing the Python import test. Do we know what could be causing this?

import: 'tiledbsoma'
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/tiledbsoma_1709578816353/test_tmp/run_test.py", line 2, in <module>
    import tiledbsoma
  File "/home/conda/feedstock_root/build_artifacts/tiledbsoma_1709578816353/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/python3.10/site-packages/tiledbsoma/__init__.py", line 146, in <module>
    from ._collection import Collection
  File "/home/conda/feedstock_root/build_artifacts/tiledbsoma_1709578816353/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/python3.10/site-packages/tiledbsoma/_collection.py", line 35, in <module>
    from . import _funcs, _tdb_handles
  File "/home/conda/feedstock_root/build_artifacts/tiledbsoma_1709578816353/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/python3.10/site-packages/tiledbsoma/_tdb_handles.py", line 39, in <module>
    from .options._soma_tiledb_context import SOMATileDBContext
  File "/home/conda/feedstock_root/build_artifacts/tiledbsoma_1709578816353/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/python3.10/site-packages/tiledbsoma/options/__init__.py", line 1, in <module>
    from ._soma_tiledb_context import SOMATileDBContext
  File "/home/conda/feedstock_root/build_artifacts/tiledbsoma_1709578816353/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/python3.10/site-packages/tiledbsoma/options/_soma_tiledb_context.py", line 14, in <module>
    from somacore import ContextBase
ImportError: cannot import name 'ContextBase' from 'somacore' (/home/conda/feedstock_root/build_artifacts/tiledbsoma_1709578816353/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/lib/python3.10/site-packages/somacore/__init__.py)
WARNING: Tests failed for tiledbsoma-py-1.7.3-py310h1bcaa23_0.conda - moving package to /home/conda/feedstock_root/build_artifacts/broken
TESTS FAILED: tiledbsoma-py-1.7.3-py310h1bcaa23_0.conda

[johnkerl]

@jdblischak this ContextBase thing is up my alley & I'll debug

[johnkerl]

• ContextBase exists in somacore 1.0.8
• ContextBase does not exist in somacore 1.0.7
• Tip of release-1.7 of TileDB-SOMA does indeed have references to ContextBase

I'll re-examine my recent backports to the release-1.7 branch of TileDB-SOMA

[johnkerl]

• https://github.com/single-cell-data/TileDB-SOMA/pull/2192 as culprit
• https://github.com/TileDB-Inc/tiledbsoma-feedstock/pull/95/commits/00575623bb7e115c7b04b2cec77f8d08fc7a6e94 as fix

[johnkerl]

See also https://github.com/single-cell-data/TileDB-SOMA/issues/2210

[johnkerl]

CI is green; this PR has done its job per our established procedure

See also notes on https://github.com/single-cell-data/TileDB-SOMA/issues/2210 that the cmake dependency in the build line here is a workaround not a fix -- build-from-source should not be happening in the first place

[johnkerl]

Running an experiment with @jdblischak to avoid the compile-from-source problem on MacOS

[jdblischak]

Good news. The recent run was successful. I checked the build logs, and they all properly find the existing libtiledbsoma in the conda env:

# linux-64
Checking: /home/conda/feedstock_root/build_artifacts/tiledbsoma_1709747283748/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/lib/libtiledbsoma.so exists: True

# osx-64
Checking: /Users/runner/miniforge3/conda-bld/tiledbsoma_1709747369908/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/libtiledbsoma.dylib exists: True

# osx-arm64
Checking: /Users/runner/miniforge3/conda-bld/tiledbsoma_1709747350467/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/libtiledbsoma.dylib exists: True

However, something weird is going on. pip is installing all the build dependencies into a temporary directory and using them during the build. This is bad since we want to be using the exact conda-forge build dependencies that we specified in the recipe.

/home/conda/feedstock_root/build_artifacts/tiledbsoma_1709747283748/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/bin/python -m pip install --no-deps . -vv
Using pip 24.0 from /home/conda/feedstock_root/build_artifacts/tiledbsoma_1709747283748/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/lib/python3.10/site-packages/pip (python 3.10)
Non-user install because user site-packages disabled
Created temporary directory: /tmp/pip-build-tracker-2izr_w9_
Initialized build tracking at /tmp/pip-build-tracker-2izr_w9_
Created build tracker: /tmp/pip-build-tracker-2izr_w9_
Entered build tracker: /tmp/pip-build-tracker-2izr_w9_
Created temporary directory: /tmp/pip-install-eqp1juuz
Created temporary directory: /tmp/pip-ephem-wheel-cache-29eiskqb
Processing /home/conda/feedstock_root/build_artifacts/tiledbsoma_1709747283748/work/apis/python
  Added file:///home/conda/feedstock_root/build_artifacts/tiledbsoma_1709747283748/work/apis/python to build tracker '/tmp/pip-build-tracker-2izr_w9_'
  Created temporary directory: /tmp/pip-build-env-9xw1xk06
  Running command pip subprocess to install build dependencies
  Installing build dependencies: started
  Using pip 24.0 from /home/conda/feedstock_root/build_artifacts/tiledbsoma_1709747283748/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/lib/python3.10/site-packages/pip (python 3.10)
  Collecting pybind11>=2.10.0 (from pybind11[global]>=2.10.0)
    Obtaining dependency information for pybind11>=2.10.0 from https://files.pythonhosted.org/packages/06/55/9f73c32dda93fa4f539fafa268f9504e83c489f460c380371d94296126cd/pybind11-2.11.1-py3-none-any.whl.metadata
    Downloading pybind11-2.11.1-py3-none-any.whl.metadata (9.5 kB)
  Collecting setuptools>=65.5.1
    Obtaining dependency information for setuptools>=65.5.1 from https://files.pythonhosted.org/packages/c0/7a/3da654f49c95d0cc6e9549a855b5818e66a917e852ec608e77550c8dc08b/setuptools-69.1.1-py3-none-any.whl.metadata
    Downloading setuptools-69.1.1-py3-none-any.whl.metadata (6.2 kB)
  Collecting wheel>=0.37.1
    Obtaining dependency information for wheel>=0.37.1 from https://files.pythonhosted.org/packages/c7/c3/55076fc728723ef927521abaa1955213d094933dc36d4a2008d5101e1af5/wheel-0.42.0-py3-none-any.whl.metadata
    Downloading wheel-0.42.0-py3-none-any.whl.metadata (2.2 kB)
  Collecting cmake>=3.21
    Obtaining dependency information for cmake>=3.21 from https://files.pythonhosted.org/packages/91/f2/93ceb9aaa7779abe5c3f4e361ba5a55b1ff0142ffb5966e7b8c1cfc0f627/cmake-3.28.3-py2.py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata
    Downloading cmake-3.28.3-py2.py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (6.3 kB)
  Collecting pybind11-global==2.11.1 (from pybind11[global]>=2.10.0)
    Obtaining dependency information for pybind11-global==2.11.1 from https://files.pythonhosted.org/packages/5b/ea/6548ee6faac053f07e3df29c7fe8207572ee38e58beb5663c02cf6d86ae0/pybind11_global-2.11.1-py3-none-any.whl.metadata
    Downloading pybind11_global-2.11.1-py3-none-any.whl.metadata (9.3 kB)
  Downloading pybind11-2.11.1-py3-none-any.whl (227 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 227.7/227.7 kB 2.0 MB/s eta 0:00:00
  Downloading setuptools-69.1.1-py3-none-any.whl (819 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 819.3/819.3 kB 8.3 MB/s eta 0:00:00
  Downloading wheel-0.42.0-py3-none-any.whl (65 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.4/65.4 kB 11.3 MB/s eta 0:00:00
  Downloading cmake-3.28.3-py2.py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (26.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 26.3/26.3 MB 45.7 MB/s eta 0:00:00
  Downloading pybind11_global-2.11.1-py3-none-any.whl (412 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 412.2/412.2 kB 46.2 MB/s eta 0:00:00
  Installing collected packages: cmake, wheel, setuptools, pybind11-global, pybind11
    Creating /tmp/pip-build-env-9xw1xk06/overlay/bin
    changing mode of /tmp/pip-build-env-9xw1xk06/overlay/bin/cmake to 755
    changing mode of /tmp/pip-build-env-9xw1xk06/overlay/bin/cpack to 755
    changing mode of /tmp/pip-build-env-9xw1xk06/overlay/bin/ctest to 755
    changing mode of /tmp/pip-build-env-9xw1xk06/overlay/bin/wheel to 755
    changing mode of /tmp/pip-build-env-9xw1xk06/overlay/bin/pybind11-config to 755
  Successfully installed cmake-3.28.3 pybind11-2.11.1 pybind11-global-2.11.1 setuptools-69.1.1 wheel-0.42.0
  Installing build dependencies: finished with status 'done'

Maybe we need to add the flag --no-build-isolation

  --no-build-isolation        Disable isolation when building a modern source
                              distribution. Build dependencies specified by
                              PEP 518 must be already installed if this option
                              is used.

[johnkerl]

Again green; closing as above

[johnkerl]

Re-running with Git hash for tip of release-1.7