search

conda-forge / mpi4py-feedstock

A conda-smithy repository for mpi4py.
BSD 3-Clause "New" or "Revised" License
4 stars 20 forks source link

OpenMPI build on osx-arm64 injects unavailable compiler into mpi4py.get_config() #56

Closed dionhaefner closed 2 years ago

dionhaefner commented 2 years ago

Solution to issue cannot be found in the documentation.

Issue

Example:

$ conda create -n mpi4py-mpich -c conda-forge mpi4py mpich
$ conda activate mpi4py-mpich
$ python -c "import mpi4py; print(mpi4py.get_config())"
{'mpicc': '/opt/miniconda3/envs/mpi4py-mpich/bin/mpicc', 'mpicxx': '/opt/miniconda3/envs/mpi4py-mpich/bin/mpicxx', 'mpifort': '/opt/miniconda3/envs/mpi4py-mpich/bin/mpifort', 'mpif90': '/opt/miniconda3/envs/mpi4py-mpich/bin/mpif90', 'mpif77': '/opt/miniconda3/envs/mpi4py-mpich/bin/mpif77'}
# all good
$ conda create -n mpi4py-ompi -c conda-forge mpi4py openmpi
$ conda activate mpi4py-ompi
$ python -c "import mpi4py; print(mpi4py.get_config())"
{'mpicc': '/Users/runner/miniforge3/conda-bld/mpi4py_1649058206187/_build_env/bin/mpicc', 'mpicxx': '/Users/runner/miniforge3/conda-bld/mpi4py_1649058206187/_build_env/bin/mpicxx', 'mpifort': '/Users/runner/miniforge3/conda-bld/mpi4py_1649058206187/_build_env/bin/mpifort', 'mpif90': '/Users/runner/miniforge3/conda-bld/mpi4py_1649058206187/_build_env/bin/mpif90', 'mpif77': '/Users/runner/miniforge3/conda-bld/mpi4py_1649058206187/_build_env/bin/mpif77'}
# all paths are from the conda-forge build environment, not the current environment

This is only the case on osx-arm64, all other platforms work as intended.

Installed packages

# packages in environment at /opt/miniconda3/envs/mpi4py-ompi:
#
# Name                    Version                   Build  Channel
bzip2                     1.0.8                h3422bc3_4    conda-forge
ca-certificates           2022.6.15            h4653dfc_0    conda-forge
libcxx                    14.0.6               h04bba0f_0    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libgfortran               5.0.0.dev0      11_0_1_hf114ba7_23    conda-forge
libgfortran5              11.0.1.dev0         hf114ba7_23    conda-forge
libzlib                   1.2.12               ha287fd2_2    conda-forge
llvm-openmp               14.0.4               hd125106_0    conda-forge
mpi                       1.0                     openmpi    conda-forge
mpi4py                    3.1.3           py310h2c02159_1    conda-forge
ncurses                   6.3                  h07bb92c_1    conda-forge
openmpi                   4.1.4              h63a3ee9_100    conda-forge
openssl                   3.0.5                h7aea29f_1    conda-forge
pip                       22.2.2             pyhd8ed1ab_0    conda-forge
python                    3.10.5          h4eee789_0_cpython    conda-forge
python_abi                3.10                    2_cp310    conda-forge
readline                  8.1.2                h46ed386_0    conda-forge
setuptools                63.4.3          py310hbe9552e_0    conda-forge
sqlite                    3.39.2               h40dfcc0_0    conda-forge
tk                        8.6.12               he1e0b03_0    conda-forge
tzdata                    2022a                h191b570_0    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xz                        5.2.5                h642e427_1    conda-forge
zlib                      1.2.12               ha287fd2_2    conda-forge

Environment info

active environment : mpi4py-ompi
    active env location : /opt/miniconda3/envs/mpi4py-ompi
            shell level : 2
       user config file : /Users/dion/.condarc
 populated config files : /Users/dion/.condarc
          conda version : 4.12.0
    conda-build version : not installed
         python version : 3.9.12.final.0
       virtual packages : __osx=12.5=0
                          __unix=0=0
                          __archspec=1=arm64
       base environment : /opt/miniconda3  (writable)
      conda av data dir : /opt/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.software.inl.gov/public/osx-arm64
                          https://conda.software.inl.gov/public/noarch
                          https://conda.anaconda.org/conda-forge/osx-arm64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /opt/miniconda3/pkgs
                          /Users/dion/.conda/pkgs
       envs directories : /opt/miniconda3/envs
                          /Users/dion/.conda/envs
               platform : osx-arm64
             user-agent : conda/4.12.0 requests/2.27.1 CPython/3.9.12 Darwin/21.6.0 OSX/12.5
                UID:GID : 501:20
             netrc file : None
           offline mode : False
dalcinl commented 2 years ago

@minrk This looks like an issue related to path relocation. mpi4py installs a mpi.cfg text file as package data, and the paths in that file are not being updated. Should the recipe use build/has_prefix_files to list that particular file? I don't get why this is happening only on macOS.

minrk commented 2 years ago

It has to do with peculiarities of openmpi and cross compilation. There shouldn't be any references to the build environment. Normally, mpicxx is only in the host environment (this is true in all cases except cross-compiled openmpi). References to the host environment are rewritten for relocatability, but any reference to the build environment in the installed files (even before packaging) is incorrect.

leofang commented 2 years ago

To add to @minrk's comment, this only happens when cross-compiling because we need MPI to exist in both build and host envs, and so only osx-arm64 has the issue (#31).

dalcinl commented 2 years ago

any reference to the build environment in the installed files (even before packaging) is incorrect.

Sorry, I'm a bit confused. Is this an mpi4py issue (because it stores the absolute path to mpicc at build time), or a recipe/build-script issue in the mpi4py-feedstock, or a conda-build problem?.

leofang commented 2 years ago

I tend to think it's something mpi4py should fix -- when dumping the path to mpicc etc to mpi.cfg it should use the host env's paths, so that conda-build can take over when rewriting the paths. But I don't know how feasible this is or if it makes sense at all, since by definition users should not use the returned mpicc paths given it's cross-compiled...

dalcinl commented 2 years ago

I tend to think it's something mpi4py should fix

Well, mpi4py picks the first mpicc command it finds in $PATH at build time. That's what makes sense if you pip install mpi4py. Upstream should not have to accommodate for the details of downstream build/packaging systems. At best, we can add a fix in the recipe's build script to replace the build path with the host path. But I have the feeling that this is a conda-build issue: why isn't the build path considered for relocation?

dionhaefner commented 2 years ago

May I interject and ask why this is only an issue for OpenMPI and not for MPICH? Is mpi4py not cross-compiled with MPICH?

minrk commented 2 years ago

Is this an mpi4py issue (because it stores the absolute path to mpicc at build time), or a recipe/build-script issue in the mpi4py-feedstock, or a conda-build problem?.

It's mostly a recipe issue, though arguably an mpi4py issue that it's not overrideable (if indeed it's not). There shouldn't be any references to the build environment in what's installed. conda-build could paper over these with likely fixes (replace build prefix with host), but they shouldn't generally happen. It also wouldn't work correctly if it were found in a binary file, because the build prefix doesn't get the same padding as the host prefix.

In general, it's not a safe assumption of packages to record where things were at build time as translatable to where they should be at runtime. Anything where you are recording the build environment like this should support an explicit override to tell the file where they will be at runtime, otherwise it's a problem for relocation.

why this is only an issue for OpenMPI and not for MPICH

The short answer is that mpicc is only in the build environment for openmpi + cross compiling.

The reason that's the case is that in mpich mpicc is a script that configures $CC, whereas openmpi's mpicc is an executable so it needs to match the build architecture and not the target architecture. That's why only openmpi needs to be in the build environment for cross compilation and not mpich.

minrk commented 2 years ago

57 does the rewriting with a bit of sed.

Arguably, the same issue is in mpi4jax where the mpicc is not overrideable. Respecting e.g. $MPICC as an override for the mpi compiler to use would solve the problem there, too.

dionhaefner commented 2 years ago

Thanks for the quick fix and advice. I think introducing that environment variable is a good idea.

dalcinl commented 2 years ago

Respecting e.g. $MPICC as an override for the mpi compiler to use would solve the problem there, too.

@dionhaefner For reference, take a look here.