search

conda-forge / openmpi-feedstock

A conda-smithy repository for openmpi.
BSD 3-Clause "New" or "Revised" License
9 stars 22 forks source link

Apple Silicon conda install leads to mpif90: Bad CPU type in executable #129

Closed joezuntz closed 8 months ago

joezuntz commented 8 months ago

Solution to issue cannot be found in the documentation.

Issue

My conda package, cosmosis, depends on MPI. The MPICH-linked version works fine everywhere, but I haven't been able to get OpenMPI to work with Apple Silicon builds.

When I try to build with OpenMPI for arm64 then the supplied mpif90 executable cannot be run and crashes with the message "Bad CPU type in executable".

I guess an intel executable is being supplied? The regular gfortran compiler works fine.

Installed packages

This is happening during the build process; here is what is printed during that process.

bzip2:                       1.0.8-h3422bc3_4           conda-forge
ca-certificates:             2023.7.22-hf0a4a13_0       conda-forge
libblas:                     3.9.0-5_h880f123_netlib    conda-forge
libcxx:                      16.0.6-h4653b0c_0          conda-forge
libffi:                      3.4.2-h3422bc3_5           conda-forge
libgfortran:                 5.0.0-13_2_0_hd922786_1    conda-forge
libgfortran5:                13.2.0-hf226fd6_1          conda-forge
liblapack:                   3.9.0-5_h880f123_netlib    conda-forge
libsqlite:                   3.43.2-h091b4b1_0          conda-forge
libzlib:                     1.2.13-h53f4e23_5          conda-forge
llvm-openmp:                 17.0.3-hcd81f8e_0          conda-forge
minuit2_standalone:          6.22.06-hbdafb3b_100       conda-forge
mpi:                         1.0-openmpi                conda-forge
ncurses:                     6.4-h7ea286d_0             conda-forge
openmpi:                     4.1.6-h526c993_101         conda-forge
openssl:                     3.1.4-h0d3ecfb_0           conda-forge
pip:                         23.3.1-pyhd8ed1ab_0        conda-forge
python:                      3.10.13-h2469fbe_0_cpython conda-forge
readline:                    8.2-h92ec313_1             conda-forge
setuptools:                  68.2.2-pyhd8ed1ab_0        conda-forge
tk:                          8.6.13-hb31c410_0          conda-forge
tzdata:                      2023c-h71feb2d_0           conda-forge
wheel:                       0.41.2-pyhd8ed1ab_0        conda-forge
xz:                          5.2.6-h57fd34a_0           conda-forge
zlib:                        1.2.13-h53f4e23_5          conda-forge
bzip2:                       1.0.8-h0d85af4_4           conda-forge
ca-certificates:             2023.7.22-h8857fd0_0       conda-forge
cctools_osx-64:              973.0.1-ha1c5b94_15        conda-forge
cctools_osx-arm64:           973.0.1-h7bb7a8e_15        conda-forge
clang:                       16.0.6-hc177806_1          conda-forge
clang-16:                    16.0.6-default_h762fdd7_1  conda-forge
clang_impl_osx-arm64:        16.0.6-ha81d56d_6          conda-forge
clang_osx-arm64:             16.0.6-h40be2b1_6          conda-forge
clangxx:                     16.0.6-default_h762fdd7_1  conda-forge
clangxx_impl_osx-arm64:      16.0.6-hf982f3f_6          conda-forge
clangxx_osx-arm64:           16.0.6-h072c24e_6          conda-forge
compiler-rt:                 16.0.6-he1888fc_1          conda-forge
compiler-rt_osx-64:          16.0.6-he1888fc_1          conda-forge
cross-python_osx-arm64:      3.10-41_cpython            conda-forge
crossenv:                    1.4.0-pyhd8ed1ab_0         conda-forge
gettext:                     0.21.1-h8a4c099_0          conda-forge
gfortran_impl_osx-64:        13.2.0-h7643cbd_1          conda-forge
gfortran_impl_osx-arm64:     12.3.0-h73c6311_1          conda-forge
gfortran_osx-arm64:          12.3.0-h64a2375_1          conda-forge
gmp:                         6.2.1-h2e338ed_0           conda-forge
icu:                         73.2-hf5e326d_0            conda-forge
isl:                         0.25-hb486fe8_0            conda-forge
ld64_osx-64:                 609-ha20a434_15            conda-forge
ld64_osx-arm64:              609-h6aa3cc8_15            conda-forge
libclang-cpp16:              16.0.6-default_h762fdd7_1  conda-forge
libcxx:                      16.0.6-hd57cbcb_0          conda-forge
libffi:                      3.4.2-h0d85af4_5           conda-forge
libgfortran-devel_osx-64:    13.2.0-h80d4556_1          conda-forge
libgfortran-devel_osx-arm64: 12.3.0-hc62be1c_1          conda-forge
libgfortran5:                13.2.0-h2873a65_1          conda-forge
libiconv:                    1.17-hac89ed1_0            conda-forge
libllvm16:                   16.0.6-he4b1e75_2          conda-forge
libsqlite:                   3.43.2-h92b6c6a_0          conda-forge
libxml2:                     2.11.5-h3346baf_1          conda-forge
libzlib:                     1.2.13-h8a1eda9_5          conda-forge
llvm-openmp:                 16.0.6-hff08bdf_0          conda-forge
llvm-tools:                  16.0.6-he4b1e75_2          conda-forge
lz4-c:                       1.9.4-hf0c8a7f_0           conda-forge
make:                        4.3-h22f3db7_1             conda-forge
mpc:                         1.3.1-h81bd1dd_0           conda-forge
mpfr:                        4.2.1-h0c69b56_0           conda-forge
ncurses:                     6.4-hf0c8a7f_0             conda-forge
openssl:                     3.1.4-hd75f5a5_0           conda-forge
popt:                        1.16-h7b079dc_2002         conda-forge
python:                      3.10.13-h00d2728_0_cpython conda-forge
readline:                    8.2-h9e318b2_1             conda-forge
rsync:                       3.2.7-h30d983d_0           conda-forge
sed:                         4.7-h3efe00b_1000          conda-forge
sigtool:                     0.1.3-h88f4db0_0           conda-forge
tapi:                        1100.0.11-h9ce4665_0       conda-forge
tk:                          8.6.13-hef22860_0          conda-forge
tzdata:                      2023c-h71feb2d_0           conda-forge
xxhash:                      0.8.0-h35c211d_3           conda-forge
xz:                          5.2.6-h775f41a_0           conda-forge
zlib:                        1.2.13-h8a1eda9_5          conda-forge
zstd:                        1.5.5-h829000d_0           conda-forge

Environment info

active environment : base
   active env location : /Users/runner/miniforge3
           shell level : 1
      user config file : /Users/runner/.condarc
populated config files : /Users/runner/miniforge3/.condarc
         conda version : 23.3.1
   conda-build version : 3.25.0
        python version : 3.10.12.final.0
      virtual packages : __archspec=1=x86_64
                         __osx=11.7.10=0
                         __unix=0=0
      base environment : /Users/runner/miniforge3  (writable)
     conda av data dir : /Users/runner/miniforge3/etc/conda
 conda av metadata url : None
          channel URLs : https://conda.anaconda.org/conda-forge/osx-64
                         https://conda.anaconda.org/conda-forge/noarch
         package cache : /Users/runner/miniforge3/pkgs
                         /Users/runner/.conda/pkgs
      envs directories : /Users/runner/miniforge3/envs
                         /Users/runner/.conda/envs
              platform : osx-64
            user-agent : conda/23.3.1 requests/2.31.0 CPython/3.10.12 Darwin/20.6.0 OSX/11.7.10
               UID:GID : 501:20
            netrc file : None
          offline mode : Fals
leofang commented 8 months ago

On conda-forge, osx-arm64 packages are cross compiled on x86-64 Azure instances. So, when cross-compiling, you also have to add openmpi or any build-time dependencies to the build section. https://conda-forge.org/docs/maintainer/knowledge_base.html#cross-compilation Also, make test can't be run because it's for native builds.

dalcinl commented 8 months ago

@joezuntz For a specific example of what @leofang explained, see this line.

beckermr commented 8 months ago

@leofang @dalcinl We're seeing x86_64 compilers pulled into build here: https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=814046&view=logs&jobId=d1321064-f6c3-56b7-0172-6d994d01c836&j=55dcca09-69c4-5f54-9c14-572421a2beca&t=e10c6994-80d2-5627-c9cb-9ef49f0ff282

We see this for mpif90 -show

x86_64-apple-darwin13.4.0-gfortran -I$BUILD_PREFIX/include -Wl,-flat_namespace -Wl...

This doesn't appear to happen with mpich. Any ideas?

jakirkham commented 8 months ago

Probably helps to link the PR: https://github.com/conda-forge/cosmosis-feedstock/pull/57

Note this PR adds the line requested above

dalcinl commented 8 months ago

@beckermr After looking at the openmpi package artifacts, everything looks fine, there is no reference to x86_64 stuff.

You are cross-compiling, right? I'm not really sure what's going on, but I think that the x86_64 compiler version does not support the -march=arm8.3-a flag.

A possible workaround you could try is to add the following line at the beginning of your build script:

export OMPI_FC=$FC
joezuntz commented 8 months ago

This is cross-compiling, yes.

Under mpich, running mpif90 -show gives:

arm64-apple-darwin20.0.0-gfortran -I$PREFIX/include -fallow-argument-mismatch -L$PREFIX/lib -Wl,-rpath,$PREFIX/lib -Wl,-commons,use_dylibs -I$PREFIX/include -I$PREFIX/include -L$PREFIX/lib -lmpifort -lmpi -lpmpi

i.e. pulling in the arm64 fortran compiler (this is line 273 of this, whereas under openmpi it gives what Matt posted above, which is line 279 of this

I'll have a go with the env var you suggested - thanks!

dalcinl commented 8 months ago

if you install (or download manually) Open MPI in Apple Silicon and then run cat $CONDA_PREFIX/share/openmpi/mpifort-wrapper-data.txt, you should see a line with compiler=arm64-apple-darwin20.0.0-gfortran. From there on, I have no idea what magic incantation is changing the compiler command under the hood. Maybe it is Open MPI's mpifort/mpif90 using some environment variable, or some other logic. If that's the case, and unless someone can provide evidence we are doing something wrong in this feedstock, you will have to ask Open MPI uptream.

joezuntz commented 8 months ago

The environment variable change seems to have fixed my issue - thanks so much!