johnkerl
commented
1 month ago Hi @chrish935 !
We've seen this before on #2580 -- although I am surprised you're seeing this even on a build from source.
I'll take a look
Open chrish935 opened 1 month ago
johnkerl
commented
1 month ago Hi @chrish935 !
We've seen this before on #2580 -- although I am surprised you're seeing this even on a build from source.
I'll take a look
eddelbuettel
commented
1 month ago @chrish935 Thanks for the detailed logs here. Your procinfo confirms that the CPU flags do not have AVX2, and the Xeon model number seems old (but a quick check at Wikipedia did not let me find a model year). We have had issues in the past (including with builds at CRAN but also users like you) where the 'having AVX2 or not' was the differentiating factor.
In the tiledb-r package we accomodate this with a test:
## on linux, we need to consider AVX2 vs non-AVX2 capabilities on the build machine
avx2 <- if (osarg == "linux" && any(grepl("avx2", readLines("/proc/cpuinfo")))) "" else "-noavx2"
## downloads are from GitHub releases
baseurl <- "https://github.com/TileDB-Inc/TileDB/releases/download"
## now switch based on macOS or Linux, using version, architecture, avx2 if needed, version and sha
dlurl <- switch(osarg,
linux = file.path(baseurl,sprintf("%s/tiledb-linux-%s%s-%s-%s.tar.gz", ver, arch, avx2, ver, sha)),
macos = file.path(baseurl,sprintf("%s/tiledb-macos-%s-%s-%s.tar.gz", ver, arch, ver, sha)),
url = urlarg)
cat("downloading", dlurl, "\n")I fear we may never have added that for the tiledbsoma packages, either Python or R, and it could well be that the Python builds you got as wheels also assume you have a newer CPU.
johnkerl
commented
1 month ago and it could well be that the Python builds you got as wheels also assume you have a newer CPU
The puzzle is that this is intending to be a build-from-source situation for TileDB-SOMA, and, cellxgene-census wheel is Python source-only (no binaries): https://pypi.org/project/cellxgene-census/#files
That said, it's not clear that this is a from-source build just yet. I'm seeing above:
> git clone https://github.com/single-cell-data/TileDB-SOMA.git # main
> python3 -m uv pip install TileDB-SOMA/apis/python/
Resolved 43 packages in 18.32s
Built tiledbsoma @ file:///scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python
Prepared 1 package in 5m 27s
Uninstalled 3 packages in 649ms
Installed 3 packages in 357ms
- somacore==1.0.11
+ somacore==1.0.13
- tiledb==0.29.1
+ tiledb==0.31.1I believe you would need
$ pip uninstall -y tiledbsoma
$ cd /your/path/to/checkout/of/TileDB-SOMA/apis/python
$ pip install .
# You can use `pip install -e .` if you want this to be editable
# Plus: edit the code, re-run the code, no rebuild step needed
# Minus: each `import tiledbsoma` is a bit slower than it would otherwise beCan you try this out?
chrish935
commented
1 month ago Sure thing.
> pip uninstall -y tiledbsoma
Found existing installation: tiledbsoma 1.11.4
Uninstalling tiledbsoma-1.11.4:
Successfully uninstalled tiledbsoma-1.11.4
> cd TileDB-SOMA/apis/python/
> pip install .
Processing /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: attrs>=22.2 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (24.2.0)
Requirement already satisfied: numba>=0.58.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.60.0)
Requirement already satisfied: numpy<2.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.26.4)
Requirement already satisfied: pandas in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (2.2.2)
Requirement already satisfied: pyarrow-hotfix in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.6)
Requirement already satisfied: scanpy>=1.9.2 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.10.2)
Requirement already satisfied: scipy in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.14.0)
Collecting somacore==1.0.13 (from tiledbsoma==1.12.0rc0.post72.dev619176400)
Using cached somacore-1.0.13-py3-none-any.whl.metadata (1.5 kB)
Collecting tiledb~=0.31.0 (from tiledbsoma==1.12.0rc0.post72.dev619176400)
Using cached tiledb-0.31.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Requirement already satisfied: typing-extensions in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (4.12.2)
Requirement already satisfied: pyarrow>=9.0.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (17.0.0)
Requirement already satisfied: anndata!=0.10.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.10.8)
Requirement already satisfied: array-api-compat!=1.5,>1.4 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.8)
Requirement already satisfied: exceptiongroup in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.2.2)
Requirement already satisfied: h5py>=3.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.11.0)
Requirement already satisfied: natsort in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (8.4.0)
Requirement already satisfied: packaging>=20.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (24.1)
Requirement already satisfied: llvmlite<0.44,>=0.43.0dev0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from numba>=0.58.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.43.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (2024.1)
Requirement already satisfied: joblib in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.4.2)
Requirement already satisfied: legacy-api-wrap>=1.4 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.4)
Requirement already satisfied: matplotlib>=3.6 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.9.1.post1)
Requirement already satisfied: networkx>=2.7 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.3)
Requirement already satisfied: patsy in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.5.6)
Requirement already satisfied: pynndescent>=0.5 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.5.13)
Requirement already satisfied: scikit-learn>=0.24 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.5.1)
Requirement already satisfied: seaborn>=0.13 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.13.2)
Requirement already satisfied: session-info in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.0.0)
Requirement already satisfied: statsmodels>=0.13 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.14.2)
Requirement already satisfied: tqdm in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (4.66.5)
Requirement already satisfied: umap-learn!=0.5.0,>=0.5 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.5.6)
Requirement already satisfied: contourpy>=1.0.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.2.1)
Requirement already satisfied: cycler>=0.10 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (4.53.1)
Requirement already satisfied: kiwisolver>=1.3.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.4.5)
Requirement already satisfied: pillow>=8 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (10.4.0)
Requirement already satisfied: pyparsing>=2.3.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.1.2)
Requirement already satisfied: six>=1.5 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.16.0)
Requirement already satisfied: threadpoolctl>=3.1.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scikit-learn>=0.24->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.5.0)
Requirement already satisfied: stdlib-list in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from session-info->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.10.0)
Using cached somacore-1.0.13-py3-none-any.whl (35 kB)
Using cached tiledb-0.31.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (15.9 MB)
Building wheels for collected packages: tiledbsoma
Building wheel for tiledbsoma (pyproject.toml) ... done
Created wheel for tiledbsoma: filename=tiledbsoma-1.12.0rc0.post72.dev619176400-cp310-cp310-linux_x86_64.whl size=1186951 sha256=5ecf4a9c0dbdc7a921346f659f62a96d61359849b9d405732b9ef6eb273bedfd
Stored in directory: /tmp/pip-ephem-wheel-cache-j4u1wxxl/wheels/17/76/6d/9ce391a1a622a7df9e5e19a14ab6dc1ce279e738d27f3a880d
Successfully built tiledbsoma
Installing collected packages: tiledb, somacore, tiledbsoma
Attempting uninstall: tiledb
Found existing installation: tiledb 0.29.1
Uninstalling tiledb-0.29.1:
Successfully uninstalled tiledb-0.29.1
Attempting uninstall: somacore
Found existing installation: somacore 1.0.11
Uninstalling somacore-1.0.11:
Successfully uninstalled somacore-1.0.11
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
cellxgene-census 1.15.0 requires tiledbsoma~=1.11.4, but you have tiledbsoma 1.12.0rc0.post72.dev619176400 which is incompatible.
Successfully installed somacore-1.0.13 tiledb-0.31.1 tiledbsoma-1.12.0rc0.post72.dev619176400Running pytest
> python3 -m pytest ./tests
========================================================================================================== test session starts ===========================================================================================================
platform linux -- Python 3.10.12, pytest-8.3.2, pluggy-1.5.0
rootdir: /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python
configfile: pyproject.toml
plugins: typeguard-4.3.0
collected 1766 items
tests/test_basic_anndata_io.py Fatal Python error: Illegal instruction
Current thread 0x00007f1ddc40b740 (most recent call first):
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/_collection.py", line 130 in create
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/io/ingest.py", line 1007 in _create_or_open_collection
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/io/ingest.py", line 505 in from_anndata
File "/scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python/tests/test_basic_anndata_io.py", line 104 in test_import_anndata
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/python.py", line 159 in pytest_pyfunc_call
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/python.py", line 1627 in runtest
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 174 in pytest_runtest_call
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 242 in <lambda>
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 341 in from_call
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 241 in call_and_report
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 132 in runtestprotocol
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 113 in pytest_runtest_protocol
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 362 in pytest_runtestloop
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 337 in _main
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 283 in wrap_session
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 330 in pytest_cmdline_main
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/config/__init__.py", line 175 in main
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/config/__init__.py", line 201 in console_main
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pytest/__main__.py", line 9 in <module>
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/runpy.py", line 86 in _run_code
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/runpy.py", line 196 in _run_module_as_main
Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, pyarrow.lib, h5py._errors, h5py.defs, h5py._objects, h5py.h5, h5py.utils, h5py.h5t, h5py.h5s, h5py.h5ac, h5py.h5p, h5py.h5r, h5py._proxy, h5py._conv, h5py.h5z, h5py.h5a, h5py.h5d, h5py.h5ds, h5py.h5g, h5py.h5i, h5py.h5f, h5py.h5fd, h5py.h5pl, h5py.h5o, h5py.h5l, h5py._selector, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.strptime, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.lib, pyarrow._compute, pandas._libs.ops, pandas._libs.hashing, pandas._libs.arrays, pandas._libs.tslib, pandas._libs.sparse, pandas._libs.internals, pandas._libs.indexing, pandas._libs.index, pandas._libs.writers, pandas._libs.join, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.groupby, pandas._libs.json, pandas._libs.parsers, pandas._libs.testing, scipy._lib._ccallback_c, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_sqrtm_triu, scipy.linalg.cython_blas, scipy.linalg._matfuncs_expm, scipy.linalg._decomp_update, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.linalg._propack._spropack, scipy.sparse.linalg._propack._dpropack, scipy.sparse.linalg._propack._cpropack, scipy.sparse.linalg._propack._zpropack, scipy.sparse.csgraph._tools, scipy.sparse.csgraph._shortest_path, scipy.sparse.csgraph._traversal, scipy.sparse.csgraph._min_spanning_tree, scipy.sparse.csgraph._flow, scipy.sparse.csgraph._matching, scipy.sparse.csgraph._reordering, numba.core.typeconv._typeconv, numba._helperlib, numba._dynfunc, numba._dispatcher, numba.core.runtime._nrt_python, numba.np.ufunc._internal, numba.experimental.jitclass._box, tiledb.libtiledb (total: 114)
Illegal instruction>>> import tiledbsoma
>>> tiledbsoma.show_package_versions()
tiledbsoma.__version__ 1.12.0rc0.post72.dev619176400
TileDB core version (libtiledbsoma) 2.25.0
python version 3.10.12.final.0
OS version Linux 3.10.0-1160.119.1.el7.x86_64BTW my apologies about
That said, it's not clear that this is a from-source build just yet. I'm seeing above:
You did indeed already show
git clone https://github.com/single-cell-data/TileDB-SOMA.git # main
python3 -m uv pip install TileDB-SOMA/apis/python/
Resolved 43 packages in 18.32s
Built tiledbsoma @ file:///scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python
Prepared 1 package in 5m 27swhich shows:
pip install with local path alreadyAs a dev I usually build a different way which produces a lot more output :) -- anyway, what you're doing here looks fine. 🤔
johnkerl
commented
1 month ago Is there a /usr/local/lib/libtiledb.* on your system?
chrish935
commented
1 month ago Doesn't appear to be, no.
> ls /usr/local/lib/libtiledb.*
ls: cannot access /usr/local/lib/libtiledb.*: No such file or directoryOK let me ask around internally for a third pair of 👀 -- thanks @chrish935 🙏
eddelbuettel
commented
1 month ago Maybe this bit would need a 'have avx2 or not' generalization like the one I show above as this will pull the 'assume AVX2' blob when we also have one for '-noavx2' in each release build:
johnkerl
commented
1 month ago @chrish935 can you pip uninstall -y tiledbsoma, then edit TileDB-SOMA/libtiledbsoma/cmake/Modules/FindTileDB_EP.cmake, then change tiledb-linux-x86_64 to tiledb-linux-x86_64-noavx2, then retry the install?
We can do this as an experiment with your help, but, look into automating in the future ...
chrish935
commented
1 month ago Still no dice.
> pip uninstall -y tiledbsoma
Found existing installation: tiledbsoma 1.12.0rc0.post72.dev619176400
Uninstalling tiledbsoma-1.12.0rc0.post72.dev619176400:
Successfully uninstalled tiledbsoma-1.12.0rc0.post72.dev619176400
> code TileDB-SOMA/libtiledbsoma/cmake/Modules/FindTileDB_EP.cmake # tiledb-linux-x86_64 -> tiledb-linux-x86_64-noavx2
> cd TileDB-SOMA/apis/python/
> pip install .
Processing /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: attrs>=22.2 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (24.2.0)
Requirement already satisfied: numba>=0.58.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.60.0)
Requirement already satisfied: numpy<2.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.26.4)
Requirement already satisfied: pandas in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (2.2.2)
Requirement already satisfied: pyarrow-hotfix in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.6)
Requirement already satisfied: scanpy>=1.9.2 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.10.2)
Requirement already satisfied: scipy in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.14.0)
Requirement already satisfied: somacore==1.0.13 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.0.13)
Requirement already satisfied: tiledb~=0.31.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.31.1)
Requirement already satisfied: typing-extensions in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (4.12.2)
Requirement already satisfied: pyarrow>=9.0.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (17.0.0)
Requirement already satisfied: anndata!=0.10.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.10.8)
Requirement already satisfied: array-api-compat!=1.5,>1.4 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.8)
Requirement already satisfied: exceptiongroup in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.2.2)
Requirement already satisfied: h5py>=3.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.11.0)
Requirement already satisfied: natsort in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (8.4.0)
Requirement already satisfied: packaging>=20.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (24.1)
Requirement already satisfied: llvmlite<0.44,>=0.43.0dev0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from numba>=0.58.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.43.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (2024.1)
Requirement already satisfied: joblib in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.4.2)
Requirement already satisfied: legacy-api-wrap>=1.4 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.4)
Requirement already satisfied: matplotlib>=3.6 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.9.1.post1)
Requirement already satisfied: networkx>=2.7 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.3)
Requirement already satisfied: patsy in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.5.6)
Requirement already satisfied: pynndescent>=0.5 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.5.13)
Requirement already satisfied: scikit-learn>=0.24 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.5.1)
Requirement already satisfied: seaborn>=0.13 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.13.2)
Requirement already satisfied: session-info in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.0.0)
Requirement already satisfied: statsmodels>=0.13 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.14.2)
Requirement already satisfied: tqdm in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (4.66.5)
Requirement already satisfied: umap-learn!=0.5.0,>=0.5 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.5.6)
Requirement already satisfied: contourpy>=1.0.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.2.1)
Requirement already satisfied: cycler>=0.10 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (4.53.1)
Requirement already satisfied: kiwisolver>=1.3.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.4.5)
Requirement already satisfied: pillow>=8 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (10.4.0)
Requirement already satisfied: pyparsing>=2.3.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.1.2)
Requirement already satisfied: six>=1.5 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.16.0)
Requirement already satisfied: threadpoolctl>=3.1.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scikit-learn>=0.24->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.5.0)
Requirement already satisfied: stdlib-list in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from session-info->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.10.0)
Building wheels for collected packages: tiledbsoma
Building wheel for tiledbsoma (pyproject.toml) ... done
Created wheel for tiledbsoma: filename=tiledbsoma-1.12.0rc0.post72.dev619176400-cp310-cp310-linux_x86_64.whl size=1186951 sha256=3d53250beb65d92a13f471ee25d91bb7bec49491f517daf8d7cceaf1fb1d60f9
Stored in directory: /tmp/pip-ephem-wheel-cache-dmi7fxyo/wheels/17/76/6d/9ce391a1a622a7df9e5e19a14ab6dc1ce279e738d27f3a880d
Successfully built tiledbsoma
Installing collected packages: tiledbsoma
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
cellxgene-census 1.15.0 requires tiledbsoma~=1.11.4, but you have tiledbsoma 1.12.0rc0.post72.dev619176400 which is incompatible.
Successfully installed tiledbsoma-1.12.0rc0.post72.dev619176400Running pytest
> python3 -m pytest ./tests
========================================================================================================== test session starts ===========================================================================================================
platform linux -- Python 3.10.12, pytest-8.3.2, pluggy-1.5.0
rootdir: /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python
configfile: pyproject.toml
plugins: typeguard-4.3.0
collected 1766 items
tests/test_basic_anndata_io.py Fatal Python error: Illegal instruction
Current thread 0x00007feec2aa7740 (most recent call first):
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/_collection.py", line 130 in create
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/io/ingest.py", line 1007 in _create_or_open_collection
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/io/ingest.py", line 505 in from_anndata
File "/scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python/tests/test_basic_anndata_io.py", line 104 in test_import_anndata
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/python.py", line 159 in pytest_pyfunc_call
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/python.py", line 1627 in runtest
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 174 in pytest_runtest_call
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 242 in <lambda>
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 341 in from_call
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 241 in call_and_report
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 132 in runtestprotocol
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 113 in pytest_runtest_protocol
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 362 in pytest_runtestloop
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 337 in _main
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 283 in wrap_session
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 330 in pytest_cmdline_main
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/config/__init__.py", line 175 in main
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/config/__init__.py", line 201 in console_main
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pytest/__main__.py", line 9 in <module>
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/runpy.py", line 86 in _run_code
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/runpy.py", line 196 in _run_module_as_main
Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, pyarrow.lib, h5py._errors, h5py.defs, h5py._objects, h5py.h5, h5py.utils, h5py.h5t, h5py.h5s, h5py.h5ac, h5py.h5p, h5py.h5r, h5py._proxy, h5py._conv, h5py.h5z, h5py.h5a, h5py.h5d, h5py.h5ds, h5py.h5g, h5py.h5i, h5py.h5f, h5py.h5fd, h5py.h5pl, h5py.h5o, h5py.h5l, h5py._selector, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.strptime, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.lib, pyarrow._compute, pandas._libs.ops, pandas._libs.hashing, pandas._libs.arrays, pandas._libs.tslib, pandas._libs.sparse, pandas._libs.internals, pandas._libs.indexing, pandas._libs.index, pandas._libs.writers, pandas._libs.join, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.groupby, pandas._libs.json, pandas._libs.parsers, pandas._libs.testing, scipy._lib._ccallback_c, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_sqrtm_triu, scipy.linalg.cython_blas, scipy.linalg._matfuncs_expm, scipy.linalg._decomp_update, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.linalg._propack._spropack, scipy.sparse.linalg._propack._dpropack, scipy.sparse.linalg._propack._cpropack, scipy.sparse.linalg._propack._zpropack, scipy.sparse.csgraph._tools, scipy.sparse.csgraph._shortest_path, scipy.sparse.csgraph._traversal, scipy.sparse.csgraph._min_spanning_tree, scipy.sparse.csgraph._flow, scipy.sparse.csgraph._matching, scipy.sparse.csgraph._reordering, numba.core.typeconv._typeconv, numba._helperlib, numba._dynfunc, numba._dispatcher, numba.core.runtime._nrt_python, numba.np.ufunc._internal, numba.experimental.jitclass._box, tiledb.libtiledb (total: 114)
Illegal instruction@johnkerl Doesn't it need an updated sha1 for the changed file too?
johnkerl
commented
1 month ago @eddelbuettel correct -- @chrish935 here https://github.com/TileDB-Inc/TileDB/releases/tag/2.25.0
[tiledb-linux-x86_64-noavx2-2.25.0-bbcbd3f.tar.gz](https://github.com/TileDB-Inc/TileDB/releases/download/2.25.0/tiledb-linux-x86_64-noavx2-2.25.0-bbcbd3f.tar.gz)-- although I'm surprised your build didn't fail with a file-not-found / curl error -- which may itself be a clue ... 🤔
I was composing the following message when @eddelbuettel 's message came in -- just now -- please disregard if the filename update helps:
@chrish935 if it's not too much trouble to ask you to do another re-run on your hardware -- in TileDB-SOMA's scripts/bld at line 121 there is
cmake_verbose="false"-- if you can change this to
cmake_verbose="true"then uninstall and re-install as you've done -- and please save and upload all output to gist.github.com and share the link here -- that will help debug 🙏
chrish935
commented
1 month ago That's just the link for the linux DOWNLOAD_URL, in the .cmake file, right? That didn't change anything (i.e., it's ultimately the same link we ended up with when initially editing TileDB-SOMA/libtiledbsoma/cmake/Modules/FindTileDB_EP.cmake).
> pip uninstall -y tiledbsoma
Found existing installation: tiledbsoma 1.12.0rc0.post72.dev619176400
Uninstalling tiledbsoma-1.12.0rc0.post72.dev619176400:
Successfully uninstalled tiledbsoma-1.12.0rc0.post72.dev619176400
> code TileDB-SOMA/libtiledbsoma/cmake/Modules/FindTileDB_EP.cmake # Changing linux DOWNLOAD_URL
> pip install TileDB-SOMA/apis/python/
Processing ./TileDB-SOMA/apis/python
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: attrs>=22.2 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (24.2.0)
Requirement already satisfied: numba>=0.58.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.60.0)
Requirement already satisfied: numpy<2.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.26.4)
Requirement already satisfied: pandas in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (2.2.2)
Requirement already satisfied: pyarrow-hotfix in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.6)
Requirement already satisfied: scanpy>=1.9.2 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.10.2)
Requirement already satisfied: scipy in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.14.0)
Requirement already satisfied: somacore==1.0.13 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (1.0.13)
Requirement already satisfied: tiledb~=0.31.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.31.1)
Requirement already satisfied: typing-extensions in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (4.12.2)
Requirement already satisfied: pyarrow>=9.0.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (17.0.0)
Requirement already satisfied: anndata!=0.10.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from tiledbsoma==1.12.0rc0.post72.dev619176400) (0.10.8)
Requirement already satisfied: array-api-compat!=1.5,>1.4 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.8)
Requirement already satisfied: exceptiongroup in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.2.2)
Requirement already satisfied: h5py>=3.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.11.0)
Requirement already satisfied: natsort in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (8.4.0)
Requirement already satisfied: packaging>=20.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from anndata!=0.10.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (24.1)
Requirement already satisfied: llvmlite<0.44,>=0.43.0dev0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from numba>=0.58.0->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.43.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (2024.1)
Requirement already satisfied: joblib in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.4.2)
Requirement already satisfied: legacy-api-wrap>=1.4 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.4)
Requirement already satisfied: matplotlib>=3.6 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.9.1.post1)
Requirement already satisfied: networkx>=2.7 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.3)
Requirement already satisfied: patsy in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.5.6)
Requirement already satisfied: pynndescent>=0.5 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.5.13)
Requirement already satisfied: scikit-learn>=0.24 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.5.1)
Requirement already satisfied: seaborn>=0.13 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.13.2)
Requirement already satisfied: session-info in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.0.0)
Requirement already satisfied: statsmodels>=0.13 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.14.2)
Requirement already satisfied: tqdm in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (4.66.5)
Requirement already satisfied: umap-learn!=0.5.0,>=0.5 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.5.6)
Requirement already satisfied: contourpy>=1.0.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.2.1)
Requirement already satisfied: cycler>=0.10 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (4.53.1)
Requirement already satisfied: kiwisolver>=1.3.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.4.5)
Requirement already satisfied: pillow>=8 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (10.4.0)
Requirement already satisfied: pyparsing>=2.3.1 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from matplotlib>=3.6->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.1.2)
Requirement already satisfied: six>=1.5 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas->tiledbsoma==1.12.0rc0.post72.dev619176400) (1.16.0)
Requirement already satisfied: threadpoolctl>=3.1.0 in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from scikit-learn>=0.24->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (3.5.0)
Requirement already satisfied: stdlib-list in /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages (from session-info->scanpy>=1.9.2->tiledbsoma==1.12.0rc0.post72.dev619176400) (0.10.0)
Building wheels for collected packages: tiledbsoma
Building wheel for tiledbsoma (pyproject.toml) ... done
Created wheel for tiledbsoma: filename=tiledbsoma-1.12.0rc0.post72.dev619176400-cp310-cp310-linux_x86_64.whl size=1186951 sha256=81c663f97995a193d9c33fa16cf54d2af67ee1873d91049a22c3eaaf264bba62
Stored in directory: /tmp/pip-ephem-wheel-cache-96yey4n7/wheels/17/76/6d/9ce391a1a622a7df9e5e19a14ab6dc1ce279e738d27f3a880d
Successfully built tiledbsoma
Installing collected packages: tiledbsoma
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
cellxgene-census 1.15.0 requires tiledbsoma~=1.11.4, but you have tiledbsoma 1.12.0rc0.post72.dev619176400 which is incompatible.
Successfully installed tiledbsoma-1.12.0rc0.post72.dev619176400Pytest
> python3 -m pytest TileDB-SOMA/apis/python/tests/
========================================================================================================== test session starts ===========================================================================================================
platform linux -- Python 3.10.12, pytest-8.3.2, pluggy-1.5.0
rootdir: /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python
configfile: pyproject.toml
plugins: typeguard-4.3.0
collected 1766 items
TileDB-SOMA/apis/python/tests/test_basic_anndata_io.py Fatal Python error: Illegal instruction
Current thread 0x00007fdebfae2740 (most recent call first):
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/_collection.py", line 130 in create
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/io/ingest.py", line 1007 in _create_or_open_collection
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/io/ingest.py", line 505 in from_anndata
File "/scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/apis/python/tests/test_basic_anndata_io.py", line 104 in test_import_anndata
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/python.py", line 159 in pytest_pyfunc_call
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/python.py", line 1627 in runtest
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 174 in pytest_runtest_call
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 242 in <lambda>
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 341 in from_call
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 241 in call_and_report
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 132 in runtestprotocol
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/runner.py", line 113 in pytest_runtest_protocol
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 362 in pytest_runtestloop
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 337 in _main
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 283 in wrap_session
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/main.py", line 330 in pytest_cmdline_main
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/config/__init__.py", line 175 in main
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/_pytest/config/__init__.py", line 201 in console_main
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/pytest/__main__.py", line 9 in <module>
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/runpy.py", line 86 in _run_code
File "/scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/runpy.py", line 196 in _run_module_as_main
Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, pyarrow.lib, h5py._errors, h5py.defs, h5py._objects, h5py.h5, h5py.utils, h5py.h5t, h5py.h5s, h5py.h5ac, h5py.h5p, h5py.h5r, h5py._proxy, h5py._conv, h5py.h5z, h5py.h5a, h5py.h5d, h5py.h5ds, h5py.h5g, h5py.h5i, h5py.h5f, h5py.h5fd, h5py.h5pl, h5py.h5o, h5py.h5l, h5py._selector, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.strptime, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.lib, pyarrow._compute, pandas._libs.ops, pandas._libs.hashing, pandas._libs.arrays, pandas._libs.tslib, pandas._libs.sparse, pandas._libs.internals, pandas._libs.indexing, pandas._libs.index, pandas._libs.writers, pandas._libs.join, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.groupby, pandas._libs.json, pandas._libs.parsers, pandas._libs.testing, scipy._lib._ccallback_c, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_sqrtm_triu, scipy.linalg.cython_blas, scipy.linalg._matfuncs_expm, scipy.linalg._decomp_update, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.linalg._propack._spropack, scipy.sparse.linalg._propack._dpropack, scipy.sparse.linalg._propack._cpropack, scipy.sparse.linalg._propack._zpropack, scipy.sparse.csgraph._tools, scipy.sparse.csgraph._shortest_path, scipy.sparse.csgraph._traversal, scipy.sparse.csgraph._min_spanning_tree, scipy.sparse.csgraph._flow, scipy.sparse.csgraph._matching, scipy.sparse.csgraph._reordering, numba.core.typeconv._typeconv, numba._helperlib, numba._dynfunc, numba._dispatcher, numba.core.runtime._nrt_python, numba.np.ufunc._internal, numba.experimental.jitclass._box, tiledb.libtiledb (total: 114)
Illegal instructionI don't see any cmake_verbose statements at line 121 of TileDB-SOMA/scripts/bld. Perhaps you mean line 14, @johnkerl ?
Lines 11 - 26 of TileDB-SOMA/scripts/bld
build="Release"
prefix=""
tiledb=""
cmake_verbose="false"
no_tiledb_deprecated="false"
while test $# != 0; do
case "$1" in
--build=*) build=$(arg "$1");;
--prefix=*) prefix=$(arg "$1");;
--tiledb=*) tiledb=$(arg "$1");;
--cmake-verbose=*) cmake_verbose=$(arg "$1");;
--no-tiledb-deprecated=*) no_tiledb_deprecated=$(arg "$1");;
esac
shift
doneLines 118 - 121 of TileDB-SOMA/scripts/bld
TILEDBSOMA_COVERAGE="${TILEDBSOMA_COVERAGE}" cmake -B build -S libtiledbsoma -DCMAKE_BUILD_TYPE=${build} ${extra_opts}
TILEDBSOMA_COVERAGE="${TILEDBSOMA_COVERAGE}" cmake --build build -j ${nproc}
TILEDBSOMA_COVERAGE="${TILEDBSOMA_COVERAGE}" cmake --build build --target install-libtiledbsoma
TILEDBSOMA_COVERAGE="${TILEDBSOMA_COVERAGE}" cmake --build build/libtiledbsoma --target build_tests -j ${nproc}Yes our thinking is that you may need this '-noavx' variant though -- from our 'releases' page -- ue the second tar.gz herecd
That being said, and while nobody is watching, you could also play a dirty trick on your machine and replace the libtiledb.so.2.25 you have with this
$ tar tvzf tiledb-linux-x86_64-noavx2-2.25.0-bbcbd3f.tar.gz | grep ".so"
-rw-r--r-- root/root 4948 2024-07-24 11:08 include/tiledb/consolidation_plan_experimental.h
-rwxr-xr-x root/root 42252696 2024-07-24 11:37 lib/libtiledb.so.2.25
lrwxrwxrwx root/root 0 2024-07-24 11:42 lib/libtiledb.so -> libtiledb.so.2.25
$ as it carries the 'no-avx2' object code that we hope avoids the 'illegal instruction' on your machine. If you replaced the library and then restarted your script or session to get this one it may succeed. Then we'd know it was the newer instruction set.
chrish935
commented
1 month ago I downloaded tiledb-linux-x86_64-noavx2-2.25.0-bbcbd3f.tar.gz, extracted libtiledb.so.2.25, & replaced all instances of libtiledb.so.2.25 with the newly extracted one before doing the usual cycle of pip uninstall, pip install, & python3 pytest. Still ran into the Illegal instruction error.
I replaced all of the libtiledb.so.2.25's that I found here
> find . -type f -name "libtiledb.so.2.25"
./TileDB-SOMA/build/externals/src/ep_tiledb/lib/libtiledb.so.2.25
./TileDB-SOMA/build/externals/install/lib/libtiledb.so.2.25
./TileDB-SOMA/dist/lib/libtiledb.so.2.25
./libtiledb.so.2.25 # This is the new one from the tar.gz downloadThis is all with the change to the .cmake file that @johnkerl originally suggested:
Original
SET(DOWNLOAD_URL "https://github.com/TileDB-Inc/TileDB/releases/download/2.25.0/tiledb-linux-x86_64-2.25.0-bbcbd3f.tar.gz")
Changed
SET(DOWNLOAD_URL "https://github.com/TileDB-Inc/TileDB/releases/download/2.25.0/tiledb-linux-x86_64-noavx2-2.25.0-bbcbd3f.tar.gz") -- note the added "noavx2"
Should I change this back at all when attempting the other suggestions?
johnkerl
commented
1 month ago @chrish935 thanks for your time on this.
What you've done so far should have worked, so in the words of Sherlock Holmes, when it's not any of thing things it can be, it has to be one of the things it can't be.
Something is not what it seems:
noavx2 tar file isn't what it seems, or/usr/local/ library is happening in a way I am not seeing, orAVX2 instruction, but some other -- gdb can perhaps reveal thatIn any event, we at TileDB need to get a system set up similar to this and debug it. I appreciate your generous time involvement but I think we're at a next step of rolling up the sleeves internally.
chrish935
commented
1 month ago Here's the output from gdb, @johnkerl .
> gdb -- python
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-120.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/bin/python3.10...Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/bin/python3.10]
(no debugging symbols found)...done.
> (gdb) r
Starting program: /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/bin/python
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Python 3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:40:32) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/readline.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so]
>>> import cellxgene_census
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_csv.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/binascii.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/zlib.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/../../libbz2.so.1.0]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_struct.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/math.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_bisect.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_random.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_sha512.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_socket.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/select.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/array.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_datetime.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/bin/../lib/libgcc_s.so.1]
Missing separate debuginfo for /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/numpy/core/../../numpy.libs/libgfortran-040039e1.so.5.0.0
Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/5b/be74eb6855e0a2c043c0bec2f484bf3e9f14c0.debug
Missing separate debuginfo for /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/numpy/core/../../numpy.libs/libquadmath-96973f99.so.0.0.0
Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/54/9b4c82347785459571c79239872ad31509dcf4.debug
[New Thread 0x7fffed03a700 (LWP 24672)]
[New Thread 0x7fffec839700 (LWP 24673)]
[New Thread 0x7fffea038700 (LWP 24674)]
[New Thread 0x7fffe7837700 (LWP 24676)]
[New Thread 0x7fffe5036700 (LWP 24677)]
[New Thread 0x7fffe2835700 (LWP 24678)]
[New Thread 0x7fffe0034700 (LWP 24679)]
[New Thread 0x7fffdd833700 (LWP 24680)]
[New Thread 0x7fffdb032700 (LWP 24681)]
[New Thread 0x7fffd8831700 (LWP 24682)]
[New Thread 0x7fffd6030700 (LWP 24683)]
[New Thread 0x7fffd382f700 (LWP 24684)]
[New Thread 0x7fffd102e700 (LWP 24685)]
[New Thread 0x7fffce82d700 (LWP 24686)]
[New Thread 0x7fffcc02c700 (LWP 24688)]
[New Thread 0x7fffc982b700 (LWP 24689)]
[New Thread 0x7fffc702a700 (LWP 24690)]
[New Thread 0x7fffc4829700 (LWP 24691)]
[New Thread 0x7fffc2028700 (LWP 24692)]
[New Thread 0x7fffbf827700 (LWP 24693)]
[New Thread 0x7fffbd026700 (LWP 24694)]
[New Thread 0x7fffba825700 (LWP 24695)]
[New Thread 0x7fffb8024700 (LWP 24697)]
[New Thread 0x7fffb5823700 (LWP 24698)]
[New Thread 0x7fffb3022700 (LWP 24699)]
[New Thread 0x7fffb0821700 (LWP 24700)]
[New Thread 0x7fffae020700 (LWP 24701)]
[New Thread 0x7fffab81f700 (LWP 24702)]
[New Thread 0x7fffa901e700 (LWP 24704)]
[New Thread 0x7fffa681d700 (LWP 24705)]
[New Thread 0x7fffa401c700 (LWP 24706)]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_pickle.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/fcntl.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_posixsubprocess.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_blake2.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/bin/../lib/libstdc++.so.6]
[New Thread 0x7fff991ff700 (LWP 25005)]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_heapq.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_zoneinfo.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/unicodedata.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/mmap.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/grp.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/cmath.cpython-310-x86_64-linux-gnu.so]
Missing separate debuginfo for /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/scipy/linalg/../../scipy.libs/libgfortran-040039e1-0352e75f.so.5.0.0
Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/5b/be74eb6855e0a2c043c0bec2f484bf3e9f14c0.debug
Missing separate debuginfo for /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/scipy/linalg/../../scipy.libs/libquadmath-96973f99-934c22de.so.0.0.0
Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/54/9b4c82347785459571c79239872ad31509dcf4.debug
[New Thread 0x7fff4f339700 (LWP 25382)]
[New Thread 0x7fff4eb38700 (LWP 25383)]
[New Thread 0x7fff4e337700 (LWP 25384)]
[New Thread 0x7fff4db36700 (LWP 25386)]
[New Thread 0x7fff4d335700 (LWP 25387)]
[New Thread 0x7fff4cb34700 (LWP 25388)]
[New Thread 0x7fff4c333700 (LWP 25389)]
[New Thread 0x7fff4bb32700 (LWP 25390)]
[New Thread 0x7fff4b331700 (LWP 25391)]
[New Thread 0x7fff4ab30700 (LWP 25392)]
[New Thread 0x7fff4a32f700 (LWP 25394)]
[New Thread 0x7fff49b2e700 (LWP 25395)]
[New Thread 0x7fff4932d700 (LWP 25396)]
[New Thread 0x7fff48b2c700 (LWP 25397)]
[New Thread 0x7fff4832b700 (LWP 25398)]
[New Thread 0x7fff47b2a700 (LWP 25400)]
[New Thread 0x7fff47329700 (LWP 25401)]
[New Thread 0x7fff46b28700 (LWP 25402)]
[New Thread 0x7fff46327700 (LWP 25403)]
[New Thread 0x7fff45b26700 (LWP 25404)]
[New Thread 0x7fff45325700 (LWP 25406)]
[New Thread 0x7fff44b24700 (LWP 25407)]
[New Thread 0x7fff44323700 (LWP 25408)]
[New Thread 0x7fff43b22700 (LWP 25409)]
[New Thread 0x7fff43321700 (LWP 25410)]
[New Thread 0x7fff42b20700 (LWP 25412)]
[New Thread 0x7fff4231f700 (LWP 25413)]
[New Thread 0x7fff41b1e700 (LWP 25414)]
[New Thread 0x7fff4131d700 (LWP 25415)]
[New Thread 0x7fff40b1c700 (LWP 25417)]
[New Thread 0x7fff4031b700 (LWP 25418)]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_multibytecodec.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/resource.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/_elementtree.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/pyexpat.cpython-310-x86_64-linux-gnu.so]
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) [in module /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so]
>>> soma = cellxgene_census.open_soma()
The "stable" release is currently 2024-07-01. Specify 'census_version="2024-07-01"' in future calls to open_soma() to ensure data consistency.
[New Thread 0x7fff33915700 (LWP 32425)]
[New Thread 0x7fff33114700 (LWP 32427)]
[New Thread 0x7fff32913700 (LWP 32429)]
[New Thread 0x7fff32112700 (LWP 32431)]
[New Thread 0x7fff31911700 (LWP 32433)]
[New Thread 0x7fff31110700 (LWP 32435)]
[New Thread 0x7fff3090f700 (LWP 32436)]
[New Thread 0x7fff3010e700 (LWP 32438)]
[New Thread 0x7fff2f90d700 (LWP 32440)]
[New Thread 0x7fff2f10c700 (LWP 32441)]
[New Thread 0x7fff2e90b700 (LWP 32443)]
[New Thread 0x7fff2e10a700 (LWP 32444)]
[New Thread 0x7fff2d909700 (LWP 32446)]
[New Thread 0x7fff2d108700 (LWP 32447)]
[New Thread 0x7fff2c907700 (LWP 32449)]
[New Thread 0x7fff2c106700 (LWP 32450)]
[New Thread 0x7fff2b905700 (LWP 32452)]
[New Thread 0x7fff2b104700 (LWP 32454)]
[New Thread 0x7fff2a903700 (LWP 32455)]
[New Thread 0x7fff2a102700 (LWP 32457)]
[New Thread 0x7fff29901700 (LWP 32459)]
[New Thread 0x7fff29100700 (LWP 32461)]
[New Thread 0x7fff288ff700 (LWP 32462)]
[New Thread 0x7fff280fe700 (LWP 32464)]
[New Thread 0x7fff278fd700 (LWP 32465)]
[New Thread 0x7fff270fc700 (LWP 32467)]
[New Thread 0x7fff268fb700 (LWP 32469)]
[New Thread 0x7fff260fa700 (LWP 32470)]
[New Thread 0x7fff258f9700 (LWP 32472)]
[New Thread 0x7fff250f8700 (LWP 32474)]
[New Thread 0x7fff248f7700 (LWP 32475)]
[New Thread 0x7fff240f6700 (LWP 32477)]
[New Thread 0x7fff238f5700 (LWP 32479)]
[New Thread 0x7fff230f4700 (LWP 32480)]
[New Thread 0x7fff228f3700 (LWP 32482)]
[New Thread 0x7fff220f2700 (LWP 32484)]
[New Thread 0x7fff218f1700 (LWP 32485)]
[New Thread 0x7fff210f0700 (LWP 32487)]
[New Thread 0x7fff208ef700 (LWP 32488)]
[New Thread 0x7fff200ee700 (LWP 32489)]
[New Thread 0x7fff1f8ed700 (LWP 32491)]
[New Thread 0x7fff1f0ec700 (LWP 32493)]
[New Thread 0x7fff1e8eb700 (LWP 32494)]
[New Thread 0x7fff1e0ea700 (LWP 32495)]
[New Thread 0x7fff1d8e9700 (LWP 32497)]
[New Thread 0x7fff1d0e8700 (LWP 32498)]
[New Thread 0x7fff1c8e7700 (LWP 32500)]
[New Thread 0x7fff1c0e6700 (LWP 32501)]
[New Thread 0x7fff1b8e5700 (LWP 32502)]
[New Thread 0x7fff1b0e4700 (LWP 32504)]
[New Thread 0x7fff1a8e3700 (LWP 32505)]
[New Thread 0x7fff1a0e2700 (LWP 32507)]
[New Thread 0x7fff198e1700 (LWP 32509)]
[New Thread 0x7fff190e0700 (LWP 32510)]
[New Thread 0x7fff188df700 (LWP 32511)]
[New Thread 0x7fff180de700 (LWP 32513)]
[New Thread 0x7fff178dd700 (LWP 32514)]
[New Thread 0x7fff170dc700 (LWP 32516)]
[New Thread 0x7fff168db700 (LWP 32517)]
[New Thread 0x7fff160da700 (LWP 32519)]
[New Thread 0x7fff158d9700 (LWP 32520)]
[New Thread 0x7fff150d8700 (LWP 32522)]
[New Thread 0x7fff148d7700 (LWP 32524)]
[New Thread 0x7fff140d6700 (LWP 32525)]
[New Thread 0x7fff136c9700 (LWP 32532)]
[New Thread 0x7fff12ec8700 (LWP 32534)]
[New Thread 0x7fff126c7700 (LWP 32536)]
[New Thread 0x7fff11ec6700 (LWP 32537)]
[New Thread 0x7fff116c5700 (LWP 32539)]
[New Thread 0x7fff10ec4700 (LWP 32541)]
[New Thread 0x7ffefbfff700 (LWP 32543)]
[New Thread 0x7ffefb7fe700 (LWP 32545)]
[New Thread 0x7ffefaffd700 (LWP 32547)]
[New Thread 0x7ffefa7fc700 (LWP 32549)]
[New Thread 0x7ffef9ffb700 (LWP 32551)]
[New Thread 0x7ffef97fa700 (LWP 32553)]
[New Thread 0x7ffef8ff9700 (LWP 32555)]
[New Thread 0x7ffed3fff700 (LWP 32557)]
[New Thread 0x7ffed37fe700 (LWP 32559)]
[New Thread 0x7ffecaffd700 (LWP 32561)]
[New Thread 0x7ffed2ffd700 (LWP 32564)]
[New Thread 0x7ffed27fc700 (LWP 33815)]
[Thread 0x7ffed27fc700 (LWP 33815) exited]
Program received signal SIGILL, Illegal instruction.
0x00007fff3ece65f4 in std::__detail::_Compiler<std::__cxx11::regex_traits<char> >::_Compiler(char const*, char const*, std::locale const&, std::regex_constants::syntax_option_type) ()
from /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/dist/lib64/libtiledbsoma.so
Missing separate debuginfos, use: debuginfo-install glibc-2.17-326.el7_9.3.x86_64 nss-mdns-0.14.1-9.el7.x86_64 numactl-devel-2.0.12-5.el7.x86_64
> (gdb) bt
#0 0x00007fff3ece65f4 in std::__detail::_Compiler<std::__cxx11::regex_traits<char> >::_Compiler(char const*, char const*, std::locale const&, std::regex_constants::syntax_option_type) ()
from /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/dist/lib64/libtiledbsoma.so
#1 0x00007fff3ece6be6 in tiledbsoma::util::rstrip_uri[abi:cxx11](std::basic_string_view<char, std::char_traits<char> >) ()
from /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/dist/lib64/libtiledbsoma.so
#2 0x00007fff3ec87e6c in tiledbsoma::SOMAGroup::SOMAGroup(OpenMode, std::basic_string_view<char, std::char_traits<char> >, std::shared_ptr<tiledbsoma::SOMAContext>, std::basic_string_view<char, std::char_traits<char> >, std::optional<std::pair<unsigned long, unsigned long> >) () from /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/dist/lib64/libtiledbsoma.so
#3 0x00007fff3ec8886c in tiledbsoma::SOMAGroup::open(OpenMode, std::basic_string_view<char, std::char_traits<char> >, std::shared_ptr<tiledbsoma::SOMAContext>, std::basic_string_view<char, std::char_traits<char> >, std::optional<std::pair<unsigned long, unsigned long> >) () from /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/dist/lib64/libtiledbsoma.so
#4 0x00007fff3ec8d841 in tiledbsoma::SOMAObject::open(std::basic_string_view<char, std::char_traits<char> >, OpenMode, std::shared_ptr<tiledbsoma::SOMAContext>, std::optional<std::pair<unsigned long, unsigned long> >, std::optional<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >) () from /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/dist/lib64/libtiledbsoma.so
#5 0x00007fff3edeb2ba in libtiledbsomacpp::load_soma_object(pybind11::module_&)::{lambda(std::basic_string_view<char, std::char_traits<char> >, OpenMode, std::shared_ptr<tiledbsoma::SOMAContext>, std::optional<std::pair<unsigned long, unsigned long> >, std::optional<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >)#1}::operator()(std::basic_string_view<char, std::char_traits<char> >, OpenMode, std::shared_ptr<tiledbsoma::SOMAContext>, std::optional<std::pair<unsigned long, unsigned long> >, std::optional<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >) const [clone .constprop.0] ()
from /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/pytiledbsoma.cpython-310-x86_64-linux-gnu.so
#6 0x00007fff3edeca95 in void pybind11::cpp_function::initialize<libtiledbsomacpp::load_soma_object(pybind11::module_&)::{lambda(std::basic_string_view<char, std::char_traits<char> >, OpenMode, std::shared_ptr<tiledbsoma::SOMAContext>, std::optional<std::pair<unsigned long, unsigned long> >, std::optional<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >)#1}, pybind11::object, std::basic_string_view<char, std::char_traits<char> >, OpenMode, std::shared_ptr<tiledbsoma::SOMAContext>, std::optional<std::pair<unsigned long, unsigned long> >, std::optional<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, pybind11::name, pybind11::scope, pybind11::sibling, pybind11::arg, pybind11::sibling, pybind11::sibling, pybind11::kw_only, pybind11::arg_v, pybind11::kw_only>(libtiledbsomacpp::load_soma_object(pybind11::module_&)::{lambda(std::basic_string_view<char, std::char_traits<char> >, OpenMode, std::shared_ptr<tiledbsoma::SOMAContext>, std::optional<std::pair<unsigned long, unsigned long> >, std::optional<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >)#1}&&, pybind11::object (*)(std::basic_string_view<char, std::char_traits<char> >, OpenMode, std::shared_ptr<tiledbsoma::SOMAContext>, std::optional<std::pair<unsigned long, unsigned long> >, std::optional<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >), pybind11::name const&, pybind11::scope const&, pybind11::sibling const&, pybind11::arg const&, pybind11::sibling const&, pybind11::sibling const&, pybind11::kw_only const&, pybind11::arg_v const&, pybind11::kw_only const&)::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call) ()
from /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/pytiledbsoma.cpython-310-x86_64-linux-gnu.so
#7 0x00007fff3ed5e8bd in pybind11::cpp_function::dispatcher(_object*, _object*, _object*) ()
from /scicomp/home-pure/kxc7/.conda/envs/cellxgene_census/lib/python3.10/site-packages/tiledbsoma/pytiledbsoma.cpython-310-x86_64-linux-gnu.so
#8 0x0000555555698516 in cfunction_call ()
#9 0x0000555555691a6b in _PyObject_MakeTpCall.localalias ()
#10 0x000055555568dc39 in _PyEval_EvalFrameDefault ()
#11 0x000055555569899c in _PyFunction_Vectorcall ()
#12 0x0000555555688850 in _PyEval_EvalFrameDefault ()
#13 0x000055555569899c in _PyFunction_Vectorcall ()
#14 0x0000555555688850 in _PyEval_EvalFrameDefault ()
#15 0x000055555569899c in _PyFunction_Vectorcall ()
#16 0x00005555556898fa in _PyEval_EvalFrameDefault ()
#17 0x000055555569899c in _PyFunction_Vectorcall ()
#18 0x0000555555688850 in _PyEval_EvalFrameDefault ()
#19 0x000055555569899c in _PyFunction_Vectorcall ()
#20 0x000055555568d142 in _PyEval_EvalFrameDefault ()
#21 0x000055555572bf90 in _PyEval_Vector ()
#22 0x000055555572bed7 in PyEval_EvalCode ()
#23 0x000055555575c42a in run_eval_code_obj ()
#24 0x0000555555757833 in run_mod ()
#25 0x000055555566d780 in PyRun_InteractiveOneObjectEx ()
#26 0x000055555566d907 in _PyRun_InteractiveLoopObject.localalias ()
#27 0x00005555555ed826 in _PyRun_AnyFileObject.cold ()
#28 0x000055555566da82 in PyRun_AnyFileExFlags.localalias ()
#29 0x00005555555eac62 in Py_RunMain.cold ()
#30 0x000055555571f527 in Py_BytesMain ()
#31 0x00007ffff6f02555 in __libc_start_main () from /lib64/libc.so.6
#32 0x000055555571f421 in _start ()I don't see any cmake_verbose statements at line 121 of TileDB-SOMA/scripts/bld. Perhaps you mean line 14, @johnkerl ?
Yes, I did.
If it's not too much to ask for you to do "one more thing" which is to re-run with that set -- I would appreciate it. This does generate a lot of output, so if you can gist it and share the link here, that would be wonderful ...
johnkerl
commented
1 month ago ... just saw your gdb content -- posts crossed! :)
I'm curious about this
Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4) and will look into that.
What I really want is to get the opcode for the instruction which is illegal and see if it decodes as AVX2, or something else -- not sure how to get that in gdb -- 👀
johnkerl
commented
1 month ago johnkerl
commented
1 month ago I think the TL;DR there is layout asm ...
chrish935
commented
1 month ago I don't see any cmake_verbose statements at line 121 of TileDB-SOMA/scripts/bld. Perhaps you mean line 14, @johnkerl ?
Yes, I did.
If it's not too much to ask for you to do "one more thing" which is to re-run with that set -- I would appreciate it. This does generate a lot of output, so if you can gist it and share the link here, that would be wonderful ...
Making this change didn't change what was written to stdout when doing pip install TileDB-SOMA/apis/python. Does it write the .cmake output somewhere else?
chrish935
commented
1 month ago Here's the output of layout asm after going through the gdb steps (i.e., up to when we encounter the Illegal instruction). There's quite a bit more here than what I'm sharing, but I'd be sharing a wall of text if I pasted the whole thing.
> |0x7fff3ece65f4 <_ZNSt8__detail9_CompilerINSt7__cxx1112regex_traitsIcEEEC2EPKcS6_RKSt6localeNSt15regex_constants18syntax_option_typeE+564> vinserti128 $0x1,%xmm1,%ymm0,%ymm0 │
│0x7fff3ece65fa <_ZNSt8__detail9_CompilerINSt7__cxx1112regex_traitsIcEEEC2EPKcS6_RKSt6localeNSt15regex_constants18syntax_option_typeE+570> mov %rax,0x180(%rbx) │
│0x7fff3ece6601 <_ZNSt8__detail9_CompilerINSt7__cxx1112regex_traitsIcEEEC2EPKcS6_RKSt6localeNSt15regex_constants18syntax_option_typeE+577> vmovdqu %ymm0,0x140(%rbx) │
│0x7fff3ece6609 <_ZNSt8__detail9_CompilerINSt7__cxx1112regex_traitsIcEEEC2EPKcS6_RKSt6localeNSt15regex_constants18syntax_option_typeE+585> vmovdqu %ymm0,0x160(%rbx) Making this change didn't change what was written to stdout when doing pip install TileDB-SOMA/apis/python. Does it write the .cmake output somewhere else?
No, it should be on the screen ...
There's quite a bit more here than what I'm sharing, but I'd be sharing a wall of text if I pasted the whole thing
I'm happy to take that as a gist, and you can share the link here! :)
|0x7fff3ece65f4 <_ZNSt8__detail9_CompilerINSt7__cxx1112regex_traitsIcEEEC2EPKcS6_RKSt6localeNSt15regex_constants18syntax_option_typeE+564> vinserti128 $0x1,%xmm1,%ymm0,%ymm0 │This is what I was looking for ... vinsert128 is indeed an AVX2 instruction ... 👀
johnkerl
commented
1 month ago For context, illegal instruction can be three things:
We've established it's the 1st, which is good to know
chrish935
commented
1 month ago Glad to hear that we're making some progress! Is there anything else that you'd like me to do, or are y'all kind of taking things from here?
johnkerl
commented
1 month ago Glad to hear that we're making some progress! Is there anything else that you'd like me to do, or are y'all kind of taking things from here?
I was going to try to take it from here until your messages this morning came in which are awesome, thanks!
We've established it's genuinely an AVX2 issue which is 💫 . What I'd need to do is get access to a non-AVX2-capable substrate and docker-run from there ... I'll ask around internally & see if anyone has such hardware.
No, it should be on the screen ...
There's quite a bit more here than what I'm sharing, but I'd be sharing a wall of text if I pasted the whole thing
For context, I spun up a Docker image (on an AVX2-equipped substrate, admittedly) and here is a gist: https://gist.github.com/johnkerl/7896c96b98d7905ab03d35e694ed806c
Notes:
johnkerl
commented
1 month ago I can see from your stack trace that the crash is happening at a place where tiledbsoma -- which you're building from source -- is calling directly into the core library, which you've got a noavx2 version of -- so there's something I don't understand. So only question is if there is somehow another copy of the core library on your system, getting found instead ...
chrish935
commented
1 month ago Here's the gist from building with a more verbose cmake: https://gist.github.com/chrish935/7c3cd3e48651b54c9fd7460a9ec5adb9.
Turns out that I forgot the -v option in my pip install call.
Note: This is with grabbing a fresh clone of the repo, so we're doing this without changing the download URL in TileDB-SOMA/libtiledbsoma/cmake/Modules/FindTileDB_EP.cmake or replacing the libtiledb.so.2.25 files. Also worth noting that this substantially increased build time compared to when the download URL was changed.
johnkerl
commented
1 month ago Thanks @chrish935 !
Turns out that I forgot the
-voption in mypip installcall.
Good point-out -- I usually take this for granted ...
Note: This is with grabbing a fresh clone of the repo, so we're doing this without changing the download URL in TileDB-SOMA/libtiledbsoma/cmake/Modules/FindTileDB_EP.cmake or replacing the libtiledb.so.2.25 files
Yes -- we definitely want to run this with (manual -- for now) edit in place so we can try to retrieve the noavx2 variant of the core library.
(As @eddelbuettel mentioned above, for the tiledb-r package there is auto-discovery of avx2 capability and it would be great for us to do the same in the TileDB-SOMA package, on both the Python and R sides. That said, that would be effectively the same result as you doing a manual edit of the cmake/Modules/FindTileDB_EP.cmake file -- and since you're still getting the illegal instruction after manually editing cmake/Modules/FindTileDB_EP.cmake, this means that our auto-discovery change would be necessary, but is evidently not sufficient -- there's something else going on as well.)
https://gist.github.com/chrish935/7c3cd3e48651b54c9fd7460a9ec5adb9#file-gistfile1-txt-L1744
And there it is -- this may be that "something else". It looks like this is the copy of the core library which has AVX2 instructions in it -- and this one is getting found when you do a from-source install.
I'll discuss internally & see what's best to do next, although I suspect an rm -rf of the build and dist directories and a fresh install attempt may be sufficient ...
Thanks for the time and the details @chrish935 !!
johnkerl
commented
1 month ago Confirming your pwd at which you git-cloned the TileDB-SOMA repo is
/scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA... and can you please share the ls -l and shasum -a 256 of /scicomp/groups-pure/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/build/externals/install/lib/libtiledb.so.2.25?
chrish935
commented
1 month ago Note: This is with grabbing a fresh clone of the repo, so we're doing this without changing the download URL in TileDB-SOMA/libtiledbsoma/cmake/Modules/FindTileDB_EP.cmake or replacing the libtiledb.so.2.25 files
Yes -- we definitely want to run this with (manual -- for now) edit in place so we can try to retrieve the
noavx2variant of the core library.
So we do want the edit in place for the DOWNLOAD_URL? The gist that I shared was made using a fresh clone of the repo without making the change. I can run everything again after making the change if that'd be preferable.
chrish935
commented
1 month ago Yeah, I cloned into the following:
> pwd
/scicomp/groups/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census
> ls -l
total 2
-rwx---r-x. 1 kxc7 users 251 Aug 9 12:01 environment.yml
-rwx---r-x. 1 kxc7 users 33 Aug 9 11:30 requirements.txt
-rwx---r-x. 1 kxc7 users 603 Aug 9 13:37 setup.sh
drwx---r-x. 15 kxc7 users 0 Aug 12 11:44 TileDB-SOMA # Cloned repoHere's the output that you asked for:
> pwd
/scicomp/groups/OID/NCEZID/DSR/BCFB/by-project/scrnagpt/chris/bin/cellxgene_census/TileDB-SOMA/build/externals/install/lib
> ls -l libtiledb.so.2.25
-rwx---r-x. 1 kxc7 users 42252696 Aug 12 11:35 libtiledb.so.2.25
> shasum -a 256 libtiledb.so.2.25
2a1bfa063b62cfe5f1b5668c1bfe0a204844e7e613cea8abce1fcb356f1490ff libtiledb.so.2.25Hmm
2a1bfa063b62cfe5f1b5668c1bfe0a204844e7e613cea8abce1fcb356f1490ff
matches neither
tiledb-linux-x86_64-2.25.0-bbcbd3f.tar.gz
nor
tiledb-linux-x86_64-noavx2-2.25.0-bbcbd3f.tar.gz
from
https://github.com/TileDB-Inc/TileDB/releases/tag/2.25.0 ...
$ ls -l *.gz
-rw-r--r--@ 1 johnkerl staff 14296301 Aug 12 13:58 tiledb-linux-x86_64-2.25.0-bbcbd3f.tar.gz
-rw-r--r--@ 1 johnkerl staff 14296287 Aug 12 13:58 tiledb-linux-x86_64-noavx2-2.25.0-bbcbd3f.tar.gz
$ shasum -a 256 *.gz
31d95ebbdf7f8a0a51b21566cd1d214f99fc0ce229ccbaa7ac2634b572cda008 tiledb-linux-x86_64-2.25.0-bbcbd3f.tar.gz
2efc83679a4a2accbededfa3bbd541f2bbe4cf1d7b5690378f630e2564659e91 tiledb-linux-x86_64-noavx2-2.25.0-bbcbd3f.tar.gzSo we do want the edit in place for the DOWNLOAD_URL? The gist that I shared was made using a fresh clone of the repo without making the change. I can run everything again after making the change if that'd be preferable.
Yes, these are the asks -- thank you 🙏 :
cmake_verbose="true" in scripts/bldtiledb-linux-x86_64-noavx2-2.25.0-bbcbd3f.tar.gzpip install -v -e apis/pythongit diff in your checkoutchrish935
commented
1 month ago Here's the gist. After making the change to the URL, build time again shortened to almost instantaneous & did not produce as much output even with the increased verbosity.
johnkerl
commented
1 month ago build time again shortened to almost instantaneous & did not produce as much output even with the increased verbosity
OK this is somehow not being a build from source then ... if it were it would be producing more output, and taking more time
I'll need to investigate this on a noavx2 system -- thank you very much for the experiments @chrish935 ...
Describe the bug Calling
cellxgene_census.open_soma()and/orpython3 -m pytest TileDB-SOMA/apis/python/testscauses anIllegal instructionerror. I've read through the issues here & see that building from source usually seems to be the resolution, but that hasn't worked for me (see below). I've also tried building from release 1.12 & release 1.11, but got the same error at the same steps in the workflow.To Reproduce
Creating environment
Building TileDB-SOMA
Running pytest
_Attempting
open_soma()_Versions (please complete the following information):
Python 3.10.12