search

conda-forge / fenics-feedstock

A conda-smithy repository for fenics.
BSD 3-Clause "New" or "Revised" License
13 stars 12 forks source link

Can't install fenics with conda or mamba #185

Open RemDelaporteMathurin opened 1 year ago

RemDelaporteMathurin commented 1 year ago

Solution to issue cannot be found in the documentation.

Issue

Recently, when trying to install fenics with 

conda install -c conda-forge fenics

or 

mamba install -c conda-forge fenics

We get the following error:

2023-09-26 11:56:16.351Z: #6 0.701 Traceback (most recent call last):
#6 0.701   File "/opt/conda/bin/mamba", line 7, in <module>
#6 0.701     from mamba.mamba import main
#6 0.701   File "/opt/conda/lib/python3.11/site-packages/mamba/mamba.py", line 49, in <module>
#6 0.702     import libmambapy as api
#6 0.702   File "/opt/conda/lib/python3.11/site-packages/libmambapy/__init__.py", line 7, in <module>
#6 0.702     raise e
#6 0.702   File "/opt/conda/lib/python3.11/site-packages/libmambapy/__init__.py", line 4, in <module>
#6 0.702     from libmambapy.bindings import *  # noqa: F401,F403
#6 0.702     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#6 0.702 ImportError: libarchive.so.13: cannot open shared object file: No such file or directory

Our Dockerfile is:

FROM mcr.microsoft.com/vscode/devcontainers/miniconda:0-3

RUN conda install -c conda-forge mamba
RUN mamba install -c conda-forge fenics  # <-- fails here

RUN apt-get update && apt-get install -y libgl1

RUN pip install festim==0.11.1 matplotlib meshio[all] ipykernel h-transport-materials==0.12.7 numpy==1.24

Installed packages

# packages in environment at /opt/conda:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             5.1                       1_gnu  
boltons                   23.0.0          py311h06a4308_0  
brotlipy                  0.7.0           py311h5eee18b_1002  
bzip2                     1.0.8                h7b6447c_0  
c-ares                    1.19.0               h5eee18b_0  
ca-certificates           2023.08.22           h06a4308_0  
certifi                   2023.7.22       py311h06a4308_0  
cffi                      1.15.1          py311h5eee18b_3  
charset-normalizer        2.0.4              pyhd3eb1b0_0  
conda                     23.7.4          py311h06a4308_0  
conda-content-trust       0.2.0           py311h06a4308_0  
conda-libmamba-solver     23.5.0          py311h06a4308_0  
conda-package-handling    2.1.0           py311h06a4308_0  
conda-package-streaming   0.8.0           py311h06a4308_0  
cryptography              41.0.3          py311hdda0065_0  
fmt                       9.1.0                hdb19cb5_0  
icu                       58.2                 he6710b0_3  
idna                      3.4             py311h06a4308_0  
jsonpatch                 1.32               pyhd3eb1b0_0  
jsonpointer               2.1                pyhd3eb1b0_0  
krb5                      1.20.1               h143b758_1  
ld_impl_linux-64          2.38                 h1181459_1  
libarchive                3.6.2                h6ac8c49_2  
libcurl                   8.1.1                h251f7ec_1  
libedit                   3.1.20221030         h5eee18b_0  
libev                     4.33                 h7f8727e_1  
libffi                    3.4.4                h6a678d5_0  
libgcc-ng                 11.2.0               h1234567_1  
libgomp                   11.2.0               h1234567_1  
libmamba                  1.4.1                h2dafd23_1  
libmambapy                1.4.1           py311h2dafd23_1  
libnghttp2                1.52.0               h2d74bed_1  
libsolv                   0.7.22               he621ea3_0  
libssh2                   1.10.0               hdbd6064_2  
libstdcxx-ng              11.2.0               h1234567_1  
libuuid                   1.41.5               h5eee18b_0  
libxml2                   2.10.3               hcbfbd50_0  
lz4-c                     1.9.4                h6a678d5_0  
ncurses                   6.4                  h6a678d5_0  
openssl                   3.0.10               h7f8727e_2  
packaging                 23.0            py311h06a4308_0  
pcre2                     10.37                he7ceb23_1  
pip                       23.1.2          py311h06a4308_0  
pluggy                    1.0.0           py311h06a4308_1  
pybind11-abi              4                    hd3eb1b0_1  
pycosat                   0.6.4           py311h5eee18b_0  
pycparser                 2.21               pyhd3eb1b0_0  
pyopenssl                 23.2.0          py311h06a4308_0  
pysocks                   1.7.1           py311h06a4308_0  
python                    3.11.4               h955ad1f_0  
readline                  8.2                  h5eee18b_0  
reproc                    14.2.4               h295c915_1  
reproc-cpp                14.2.4               h295c915_1  
requests                  2.31.0          py311h06a4308_0  
ruamel.yaml               0.17.21         py311h5eee18b_0  
setuptools                67.8.0          py311h06a4308_0  
six                       1.16.0             pyhd3eb1b0_1  
sqlite                    3.41.2               h5eee18b_0  
tk                        8.6.12               h1ccaba5_0  
toolz                     0.12.0          py311h06a4308_0  
tqdm                      4.65.0          py311h92b7b1e_0  
tzdata                    2023c                h04d1e81_0  
urllib3                   1.26.16         py311h06a4308_0  
wheel                     0.38.4          py311h06a4308_0  
xz                        5.4.2                h5eee18b_0  
yaml-cpp                  0.7.0                h295c915_1  
zlib                      1.2.13               h5eee18b_0  
zstandard                 0.19.0          py311h5eee18b_0  
zstd                      1.5.5                hc292b87_0

Environment info

active environment : None
            shell level : 0
       user config file : /home/vscode/.condarc
 populated config files : 
          conda version : 23.7.4
    conda-build version : not installed
         python version : 3.11.4.final.0
       virtual packages : __archspec=1=x86_64
                          __glibc=2.31=0
                          __linux=5.15.0=0
                          __unix=0=0
       base environment : /opt/conda  (writable)
      conda av data dir : /opt/conda/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/conda/pkgs
                          /home/vscode/.conda/pkgs
       envs directories : /opt/conda/envs
                          /home/vscode/.conda/envs
               platform : linux-64
             user-agent : conda/23.7.4 requests/2.31.0 CPython/3.11.4 Linux/5.15.0-1041-azure debian/11 glibc/2.31
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False
RemDelaporteMathurin commented 1 year ago

@jhdark

minrk commented 1 year ago

This is an error initializing mamba, so definitely a failed install of mamba and/or conda (specifically, mamba was installed but a compatible libarchive dependency is not installed). So it's the conda install mamba line that fails, but the second line is where the error occurs. I imagine you'd see this same error with mamba install <anything>.

This dockerfile also fails with the same error:

FROM mcr.microsoft.com/vscode/devcontainers/miniconda:0-3

RUN conda install -c conda-forge mamba
RUN mamba install -c conda-forge mamba # should be a no-op

My guess is it's the mixing of conda-forge and defaults channels that results from installing a single package form conda-forge when everything else has come from defaults. Try adding conda upgrade -c conda-forge --all before the conda install mamba step.

jhdark commented 1 year ago

Hey @minrk , I tried adding the conda upgrade -c conda-forge --all but doesn't seem to fix the issue. However, there is an error that comes after the RUN conda install -c conda-forge mamba line which is:

Error while loading conda entry point: conda-libmamba-solver (libarchive.so.13: cannot open shared object file: No such file or directory)

I tried installing this package manually with RUN conda install conda-libmamba-solver, but this didn't help either. Not sure if this helps narrow down the issue. I understand this has nothing to do with fenics, but if you have any other suggestions to try that would be great

minrk commented 1 year ago

Sorry, you need --strict-channel-prioirity to trade out all of the defaults packages for conda-forge ones:

FROM mcr.microsoft.com/vscode/devcontainers/miniconda:0-3

RUN conda upgrade --strict-channel-priority -c conda-forge --all
RUN conda install -c conda-forge conda mamba
RUN mamba install -c conda-forge fenics

But even that doesn't result in a working mamba, which I don't really understand, since it's now failing with:

0.548       File "/opt/conda/lib/python3.11/site-packages/mamba/mamba.py", line 877, in _wrapped_main
0.548         configure_parser_repoquery(p._subparsers._group_actions[0])
0.548       File "/opt/conda/lib/python3.11/site-packages/mamba/mamba.py", line 805, in configure_parser_repoquery
0.548         p = sub_parsers.add_parser(
0.548             ^^^^^^^^^^^^^^^^^^^^^^^
0.548       File "/opt/conda/lib/python3.11/argparse.py", line 1192, in add_parser
0.548         raise ArgumentError(self, _('conflicting subparser: %s') % name)
0.548     argparse.ArgumentError: argument COMMAND: conflicting subparser: repoquery

which I think means that conda and/or mamba have not actually been installed properly.

Downgrading conda to 22.x resolves the issue, I don't understand why, because when I install the same versions that don't work in an isolated env, I can't reproduce the problem. So you can pin that for now:

FROM mcr.microsoft.com/vscode/devcontainers/miniconda:0-3

RUN conda upgrade --strict-channel-priority -c conda-forge --all
RUN conda install -c conda-forge conda==22.* mamba
RUN mamba install -c conda-forge fenics

Essentially, what this is doing is removing all conda packages and replacing them with ones from conda-forge, so you might be better off using a lighter base image and installing miniforge, since that should avoid having the unused files from the original miniconda install (which is all of them) in the image layer history.

minrk commented 1 year ago

fwiw, I found that it's a very recent regression in conda-libmamba-solver: https://github.com/conda/conda-libmamba-solver/issues/303, so that's what should be pinned back, not conda itself:


FROM mcr.microsoft.com/vscode/devcontainers/miniconda:0-3

RUN conda upgrade --strict-channel-priority -c conda-forge --all
RUN conda install -c conda-forge 'conda-libmamba-solver<23.9' mamba
RUN mamba install -y -c conda-forge fenics
jhdark commented 1 year ago

Ok this is brilliant, it works now! thank you so much

RemDelaporteMathurin commented 1 year ago

@minrk there is some more explanation here https://github.com/conda/conda/issues/13148#issuecomment-1735930256