conda install problem w/python3.10: `libffi.so.7: cannot open shared object file: No such file or directory`

[shenwei356]

Can't wait to try the new version!

$ mamba install sourmash  # v4.5.0

$ sourmash -v
Traceback (most recent call last):
  File "/home/shenwei/app/miniconda3/envs/kmcp/bin/sourmash", line 7, in <module>
    from sourmash.__main__ import main
  File "/home/shenwei/app/miniconda3/envs/kmcp/lib/python3.10/site-packages/sourmash/__init__.py", line 32, in <module>
    from ._lowlevel import ffi, lib
  File "/home/shenwei/app/miniconda3/envs/kmcp/lib/python3.10/site-packages/sourmash/_lowlevel.py", line 5, in <module>
    from sourmash._lowlevel__ffi import ffi
  File "/home/shenwei/app/miniconda3/envs/kmcp/lib/python3.10/site-packages/sourmash/_lowlevel__ffi.py", line 2, in <module>
    import _cffi_backend
ImportError: libffi.so.7: cannot open shared object file: No such file or directory

# oops

Actually, libffi was installed, but the version seems too high. It needs to be specified.

$ fd libffi | grep kmcp | grep libffi
./envs/kmcp/lib/libffi.so.8

$ fd libffi | grep pkg | grep libffi.so.7
./pkgs/libffi-3.3-h58526e2_2/lib/libffi.so.7

$ fd libffi | grep pkg | grep libffi.so.8
./pkgs/libffi-3.4.2-h7f98852_5/lib/libffi.so.8

# centos 7
$ fd libffi /lib64/
/lib64/libffi.so.6

Then I tried:

$ mamba install libffi=3.3
Problem: package python-3.10.6-ha86cf86_0_cpython requires libffi >=3.4.2,<3.5.0a0, but none of the providers can be installed

OK, try Python 3.9:

$ mamba   create -n kmcp2 python==3.9

The following NEW packages will be INSTALLED:

  libffi             conda-forge/linux-64::libffi-3.3-h58526e2_2

# looks promissing!

$ mamba install sourmash

$ sourmash -v
sourmash 4.5.0

It works!

Hope this helps!

Wei

[ctb]

thanks @shenwei356!

I checked and I can't replicate this on my Linux machine - when I do:

mamba create -n smash450 python=3.10 sourmash

it seems to work 🤔

Based on the message package python-3.10.6-ha86cf86_0_cpython requires libffi >=3.4.2,<3.5.0a0 I wonder if python3.10.6 is specifying a bad CFFI?

hmm.

mamba list gives:

...
cffi                      1.15.1          py310h255011f_0    conda-forge
...
libffi                    3.4.2                h7f98852_5    conda-forge
...
python                    3.10.6          ha86cf86_0_cpython    conda-forge
...

so it looks like I have the same versions of all the things.

Weird. Will keep digging!

[shenwei356]

I just created a fresh env, it still failed.

details

$ mamba create -n smash450 python=3.10 sourmash Getting conda-forge linux-64 Getting conda-forge noarch Getting bioconda linux-64 Getting bioconda noarch Getting pkgs/main linux-64 Getting pkgs/main noarch Getting pkgs/r linux-64 Getting pkgs/r noarch Looking for: ['python 3.10.*', 'sourmash'] 290370 packages in https://conda.anaconda.org/conda-forge/linux-64 103732 packages in https://conda.anaconda.org/conda-forge/noarch 45824 packages in https://conda.anaconda.org/bioconda/linux-64 38095 packages in https://conda.anaconda.org/bioconda/noarch 26771 packages in https://repo.anaconda.com/pkgs/main/linux-64 4705 packages in https://repo.anaconda.com/pkgs/main/noarch 8442 packages in https://repo.anaconda.com/pkgs/r/linux-64 9492 packages in https://repo.anaconda.com/pkgs/r/noarch ## Package Plan ## environment location: /home/shenwei/app/miniconda3/envs/smash450 added / updated specs: - python=3.10 - sourmash The following packages will be downloaded: package | build ---------------------------|----------------- bz2file-0.98 | py310h06a4308_1 14 KB matplotlib-base-3.5.3 | py310h8d5ebf3_2 7.4 MB conda-forge pyparsing-3.0.9 | py310h06a4308_0 153 KB ------------------------------------------------------------ Total: 7.5 MB The following NEW packages will be INSTALLED: _libgcc_mutex conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge _openmp_mutex conda-forge/linux-64::_openmp_mutex-4.5-2_gnu bitstring conda-forge/noarch::bitstring-3.1.9-pyhd8ed1ab_0 brotli pkgs/main/linux-64::brotli-1.0.9-h5eee18b_7 brotli-bin pkgs/main/linux-64::brotli-bin-1.0.9-h5eee18b_7 bz2file pkgs/main/linux-64::bz2file-0.98-py310h06a4308_1 bzip2 conda-forge/linux-64::bzip2-1.0.8-h7f98852_4 ca-certificates pkgs/main/linux-64::ca-certificates-2022.07.19-h06a4308_0 cachetools conda-forge/noarch::cachetools-4.2.4-pyhd8ed1ab_0 certifi pkgs/main/linux-64::certifi-2022.6.15-py310h06a4308_0 cffi pkgs/main/linux-64::cffi-1.15.1-py310h74dc2b5_0 cycler pkgs/main/noarch::cycler-0.11.0-pyhd3eb1b0_0 deprecation pkgs/main/noarch::deprecation-2.1.0-pyhd3eb1b0_0 fonttools conda-forge/linux-64::fonttools-4.37.1-py310h5764c6d_0 freetype conda-forge/linux-64::freetype-2.12.1-hca18f0e_0 jpeg conda-forge/linux-64::jpeg-9e-h166bdaf_2 kiwisolver conda-forge/linux-64::kiwisolver-1.4.4-py310hbf28c38_0 lcms2 pkgs/main/linux-64::lcms2-2.12-h3be6417_0 ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1 lerc conda-forge/linux-64::lerc-4.0.0-h27087fc_0 libblas conda-forge/linux-64::libblas-3.9.0-16_linux64_openblas libbrotlicommon pkgs/main/linux-64::libbrotlicommon-1.0.9-h5eee18b_7 libbrotlidec pkgs/main/linux-64::libbrotlidec-1.0.9-h5eee18b_7 libbrotlienc pkgs/main/linux-64::libbrotlienc-1.0.9-h5eee18b_7 libcblas conda-forge/linux-64::libcblas-3.9.0-16_linux64_openblas libdeflate conda-forge/linux-64::libdeflate-1.13-h166bdaf_0 libffi conda-forge/linux-64::libffi-3.4.2-h7f98852_5 libgcc-ng conda-forge/linux-64::libgcc-ng-12.1.0-h8d9b700_16 libgfortran-ng conda-forge/linux-64::libgfortran-ng-12.1.0-h69a702a_16 libgfortran5 conda-forge/linux-64::libgfortran5-12.1.0-hdcd56e2_16 libgomp conda-forge/linux-64::libgomp-12.1.0-h8d9b700_16 liblapack conda-forge/linux-64::liblapack-3.9.0-16_linux64_openblas libnsl conda-forge/linux-64::libnsl-2.0.0-h7f98852_0 libopenblas conda-forge/linux-64::libopenblas-0.3.21-pthreads_h78a6416_3 libpng conda-forge/linux-64::libpng-1.6.37-h753d276_4 libsqlite conda-forge/linux-64::libsqlite-3.39.3-h753d276_0 libstdcxx-ng conda-forge/linux-64::libstdcxx-ng-12.1.0-ha89aaad_16 libtiff conda-forge/linux-64::libtiff-4.4.0-h0e0dad5_3 libuuid conda-forge/linux-64::libuuid-2.32.1-h7f98852_1000 libwebp-base conda-forge/linux-64::libwebp-base-1.2.4-h166bdaf_0 libxcb conda-forge/linux-64::libxcb-1.13-h7f98852_1004 libzlib conda-forge/linux-64::libzlib-1.2.12-h166bdaf_2 matplotlib-base conda-forge/linux-64::matplotlib-base-3.5.3-py310h8d5ebf3_2 munkres pkgs/main/noarch::munkres-1.1.4-py_0 ncurses pkgs/main/linux-64::ncurses-6.3-h5eee18b_3 numpy conda-forge/linux-64::numpy-1.23.2-py310h53a5b5f_0 openjpeg conda-forge/linux-64::openjpeg-2.5.0-h7d73246_1 openssl conda-forge/linux-64::openssl-3.0.5-h166bdaf_2 packaging pkgs/main/noarch::packaging-21.3-pyhd3eb1b0_0 pillow conda-forge/linux-64::pillow-9.2.0-py310hbd86126_2 pthread-stubs conda-forge/linux-64::pthread-stubs-0.4-h36c2ea0_1001 pycparser pkgs/main/noarch::pycparser-2.21-pyhd3eb1b0_0 pyparsing pkgs/main/linux-64::pyparsing-3.0.9-py310h06a4308_0 python conda-forge/linux-64::python-3.10.6-ha86cf86_0_cpython python-dateutil pkgs/main/noarch::python-dateutil-2.8.2-pyhd3eb1b0_0 python_abi conda-forge/linux-64::python_abi-3.10-2_cp310 readline pkgs/main/linux-64::readline-8.1.2-h7f8727e_1 scipy conda-forge/linux-64::scipy-1.9.1-py310hdfbd76f_0 screed conda-forge/noarch::screed-1.0.5-pyhd8ed1ab_1 setuptools conda-forge/linux-64::setuptools-65.3.0-py310hff52083_0 six pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_1 sourmash bioconda/noarch::sourmash-4.5.0-hdfd78af_0 sourmash-minimal conda-forge/linux-64::sourmash-minimal-4.5.0-py310h3392aa1_0 tk pkgs/main/linux-64::tk-8.6.12-h1ccaba5_0 tzdata conda-forge/noarch::tzdata-2022c-h191b570_0 unicodedata2 conda-forge/linux-64::unicodedata2-14.0.0-py310h5764c6d_1 xorg-libxau conda-forge/linux-64::xorg-libxau-1.0.9-h7f98852_0 xorg-libxdmcp conda-forge/linux-64::xorg-libxdmcp-1.1.3-h7f98852_0 xz conda-forge/linux-64::xz-5.2.6-h166bdaf_0 zlib conda-forge/linux-64::zlib-1.2.12-h166bdaf_2 zstd conda-forge/linux-64::zstd-1.5.2-h6239696_4 Proceed ([y]/n)? y Downloading and Extracting Packages pyparsing-3.0.9 | 153 KB | ################################################################################################################################################# | 100% matplotlib-base-3.5. | 7.4 MB | ################################################################################################################################################# | 100% bz2file-0.98 | 14 KB | ################################################################################################################################################# | 100% Preparing transaction: done Verifying transaction: done Executing transaction: done # # To activate this environment, use # # $ conda activate smash450 # # To deactivate an active environment, use # # $ conda deactivate $ conda activate smash450 (smash450) [shenwei@gys-svr ~]$ sourmash -v Traceback (most recent call last): File "/home/shenwei/app/miniconda3/envs/smash450/bin/sourmash", line 7, in from sourmash.__main__ import main File "/home/shenwei/app/miniconda3/envs/smash450/lib/python3.10/site-packages/sourmash/__init__.py", line 32, in from ._lowlevel import ffi, lib File "/home/shenwei/app/miniconda3/envs/smash450/lib/python3.10/site-packages/sourmash/_lowlevel.py", line 5, in from sourmash._lowlevel__ffi import ffi File "/home/shenwei/app/miniconda3/envs/smash450/lib/python3.10/site-packages/sourmash/_lowlevel__ffi.py", line 2, in import _cffi_backend ImportError: libffi.so.7: cannot open shared object file: No such file or directory

BTW, The cffi is slightly different in my env; it's installed from pypi.

$ mamba list | grep ffi
cffi                      1.15.1                   pypi_0    pypi
libffi                    3.4.2                h7f98852_5    conda-forge
(kmcp) [shenwei@gys-svr gtdb202]$ mamba list | grep python
ipython                   8.4.0                    pypi_0    pypi
ipython_genutils          0.2.0                      py_1    conda-forge
msgpack-python            1.0.4           py310hbf28c38_0    conda-forge
python                    3.10.6          ha86cf86_0_cpython    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
python_abi                3.10                    2_cp310    conda-forge

[ctb]

thank you! my current guess is this is a linux-64-specific issue, will continue to work on it!!

[luizirber]

This is fairly old, but is it still an issue, or can we close it?