Error installing AMUSE prerequisites (lapack, mpi4py)

[fredt00]

I've been trying to install AMUSE on my new M2 MacBook following the installation instructions. I got stalled when wheels for mpi4py failed to build. Looking back I realised that MacPorts had failed to install lapack. I tried to install lapack again and got the following output in terminal:

(Amuse-env) frederickthompson@asmaclap75 amuse % sudo port install lapack Password: ---> Computing dependencies for lapack ---> Fetching archive for lapack ---> Attempting to fetch lapack-3.11.0_0+gfortran.darwin_23.arm64.tbz2 from https://packages.macports.org/lapack ---> Attempting to fetch lapack-3.11.0_0+gfortran.darwin_23.arm64.tbz2 from https://fra.de.packages.macports.org/lapack ---> Attempting to fetch lapack-3.11.0_0+gfortran.darwin_23.arm64.tbz2 from https://mse.uk.packages.macports.org/lapack ---> Building lapack Error: Failed to build lapack: command execution failed Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_math_lapack/lapack/main.log for details. Error: Follow https://guide.macports.org/#project.tickets if you believe there is a bug. Error: Processing of port lapack failed

I had a look at main.log (see below) as suggested but didn't find it very helpful. What am I doing wrong here, and is lapack even the source of the issue?

Thanks in advance, Fred

main.log

[rieder]

Hi Fred, lapack is not needed for mpi4py, so I don't think this is the root of the issue. Macports is notorious for having issues on the new macOS just after its release, this may well be related to your issue - I have not checked this configuration.

[LourensVeen]

mpi4py depends only on a development install of MPI being available, so that is what you need there. Perhaps MacPorts also failed to install OpenMPI or MPICH?

[rieder]

If you're willing/able to try Homebrew rather than Macports, that might help (at least, that is my current working environment). Instructions for Homebrew (replacing those for Macports) are as below (still need to add this to the docs):

brew install gcc hdf5 netcdf cmake
brew install open-mpi --build-from-source --cc=gcc-13
brew install gsl gmp mpfr fftw openblas lapack

# lines below are needed to ensure AMUSE will pick the right compiler
# rather than the XCode compiler which doesn't play nice with AMUSE
sudo ln -s /opt/homebrew/bin/gcc-13 /opt/homebrew/bin/gcc
sudo ln -s /opt/homebrew/bin/g++-13 /opt/homebrew/bin/g++

Then install AMUSE according to the docs

[fredt00]

Thank you so much for the quick response! I've heard Macports and Homebrew don't get on well. Do I need to completely uninstall Macports and everything I installed with it first? I don't want to break other things by doing this

[rieder]

Hi, Yes, unfortunately you can't have both Macports and Homebrew on the same machine - hence, I now can't test Macports...

[rieder]

but as @LourensVeen suggested, could you first check if your OpenMPI/Mpich installation is working? This is probably what is causing mpi4py to fail.

[fredt00]

So after openMPI didn't work, I tried Mpich and they both install ok. I only have one active though:

(Amuse-env) frederickthompson@asmaclap75 amuse % port select --list mpi Available versions for mpi: mpich-gcc12-fortran none openmpi-gcc12-fortran (active)

Part of the output when I try to install mpi4py gives the mpi configuration:

running build_ext MPI configuration: [mpi] from 'mpi.cfg' MPI C compiler: /opt/local/bin/mpicc MPI C++ compiler: /opt/local/bin/mpicxx MPI F compiler: /opt/local/bin/mpifort MPI F90 compiler: /opt/local/bin/mpif90 MPI F77 compiler: /opt/local/bin/mpif77

[fredt00]

So I uninstalled all of Macports and used home-brew instead as instructed. I'm still getting a very similar error when I try to install mpi4py to a new venv:

(Amuse-env) frederickthompson@asmaclap75 amuse % pip install mpi4py --no-cache-dir --no-build-isolation --no-clean -v Using pip 23.3.1 from /Users/frederickthompson/amuse/Amuse-env/lib/python3.9/site-packages/pip (python 3.9) Collecting mpi4py Downloading mpi4py-3.1.5.tar.gz (2.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 4.7 MB/s eta 0:00:00 Running command Preparing metadata (pyproject.toml) running dist_info creating /private/var/folders/ft/b5bx87sx2gn8sq403f33mvkw0000gq/T/pip-modern-metadata-fi5gyy8f/mpi4py.egg-info writing /private/var/folders/ft/b5bx87sx2gn8sq403f33mvkw0000gq/T/pip-modern-metadata-fi5gyy8f/mpi4py.egg-info/PKG-INFO writing dependency_links to /private/var/folders/ft/b5bx87sx2gn8sq403f33mvkw0000gq/T/pip-modern-metadata-fi5gyy8f/mpi4py.egg-info/dependency_links.txt writing top-level names to /private/var/folders/ft/b5bx87sx2gn8sq403f33mvkw0000gq/T/pip-modern-metadata-fi5gyy8f/mpi4py.egg-info/top_level.txt writing manifest file '/private/var/folders/ft/b5bx87sx2gn8sq403f33mvkw0000gq/T/pip-modern-metadata-fi5gyy8f/mpi4py.egg-info/SOURCES.txt' reading manifest file '/private/var/folders/ft/b5bx87sx2gn8sq403f33mvkw0000gq/T/pip-modern-metadata-fi5gyy8f/mpi4py.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' adding license file 'LICENSE.rst' writing manifest file '/private/var/folders/ft/b5bx87sx2gn8sq403f33mvkw0000gq/T/pip-modern-metadata-fi5gyy8f/mpi4py.egg-info/SOURCES.txt' creating '/private/var/folders/ft/b5bx87sx2gn8sq403f33mvkw0000gq/T/pip-modern-metadata-fi5gyy8f/mpi4py-3.1.5.dist-info' Preparing metadata (pyproject.toml) ... done Building wheels for collected packages: mpi4py Running command Building wheel for mpi4py (pyproject.toml) running bdist_wheel running build running build_src running build_py creating build creating build/lib.macosx-10.9-universal2-cpython-39 creating build/lib.macosx-10.9-universal2-cpython-39/mpi4py copying src/mpi4py/run.py -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py copying src/mpi4py/init.py -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py copying src/mpi4py/bench.py -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py copying src/mpi4py/main.py -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py creating build/lib.macosx-10.9-universal2-cpython-39/mpi4py/futures copying src/mpi4py/futures/_base.py -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/futures copying src/mpi4py/futures/server.py -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/futures copying src/mpi4py/futures/init.py -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/futures copying src/mpi4py/futures/_core.py -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/futures copying src/mpi4py/futures/pool.py -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/futures copying src/mpi4py/futures/aplus.py -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/futures copying src/mpi4py/futures/main.py -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/futures copying src/mpi4py/futures/_lib.py -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/futures creating build/lib.macosx-10.9-universal2-cpython-39/mpi4py/util copying src/mpi4py/util/pkl5.py -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/util copying src/mpi4py/util/dtlib.py -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/util copying src/mpi4py/util/init.py -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/util copying src/mpi4py/py.typed -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py copying src/mpi4py/main.pyi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py copying src/mpi4py/init.pyi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py copying src/mpi4py/run.pyi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py copying src/mpi4py/bench.pyi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py copying src/mpi4py/MPI.pyi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py copying src/mpi4py/dl.pyi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py copying src/mpi4py/init.pxd -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py copying src/mpi4py/libmpi.pxd -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py copying src/mpi4py/MPI.pxd -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py creating build/lib.macosx-10.9-universal2-cpython-39/mpi4py/include creating build/lib.macosx-10.9-universal2-cpython-39/mpi4py/include/mpi4py copying src/mpi4py/include/mpi4py/mpi4py.MPI.h -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/include/mpi4py copying src/mpi4py/include/mpi4py/mpi4py.MPI_api.h -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/include/mpi4py copying src/mpi4py/include/mpi4py/mpi4py.h -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/include/mpi4py copying src/mpi4py/include/mpi4py/mpi4py.i -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/include/mpi4py copying src/mpi4py/include/mpi4py/mpi.pxi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/include/mpi4py copying src/mpi4py/futures/main.pyi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/futures copying src/mpi4py/futures/init.pyi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/futures copying src/mpi4py/futures/_core.pyi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/futures copying src/mpi4py/futures/aplus.pyi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/futures copying src/mpi4py/futures/server.pyi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/futures copying src/mpi4py/futures/pool.pyi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/futures copying src/mpi4py/futures/_lib.pyi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/futures copying src/mpi4py/util/init.pyi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/util copying src/mpi4py/util/dtlib.pyi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/util copying src/mpi4py/util/pkl5.pyi -> build/lib.macosx-10.9-universal2-cpython-39/mpi4py/util running build_clib MPI configuration: [mpi] from 'mpi.cfg' MPI C compiler: /opt/homebrew/bin/mpicc MPI C++ compiler: /opt/homebrew/bin/mpicxx MPI F compiler: /opt/homebrew/bin/mpifort MPI F90 compiler: /opt/homebrew/bin/mpif90 MPI F77 compiler: /opt/homebrew/bin/mpif77 checking for library 'lmpe' ... /opt/homebrew/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -c _configtest.c -o _configtest.o gcc-13: warning: this compiler does not support x86 ('-arch' option ignored) gcc-13: error: unrecognized command-line option '-iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders' gcc-13: error: unrecognized command-line option '-iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers' failure. removing: _configtest.c _configtest.o building 'mpe' dylib library creating build/temp.macosx-10.9-universal2-cpython-39 creating build/temp.macosx-10.9-universal2-cpython-39/src creating build/temp.macosx-10.9-universal2-cpython-39/src/lib-pmpi /opt/homebrew/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -c src/lib-pmpi/mpe.c -o build/temp.macosx-10.9-universal2-cpython-39/src/lib-pmpi/mpe.o gcc-13: warning: this compiler does not support x86 ('-arch' option ignored) gcc-13: error: unrecognized command-line option '-iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders' gcc-13: error: unrecognized command-line option '-iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers' warning: build_clib: command '/opt/homebrew/bin/mpicc' failed with exit code 1

warning: build_clib: building optional library "mpe" failed

checking for library 'vt-mpi' ... /opt/homebrew/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -c _configtest.c -o _configtest.o gcc-13: warning: this compiler does not support x86 ('-arch' option ignored) gcc-13: error: unrecognized command-line option '-iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders' gcc-13: error: unrecognized command-line option '-iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers' failure. removing: _configtest.c _configtest.o checking for library 'vt.mpi' ... /opt/homebrew/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -c _configtest.c -o _configtest.o gcc-13: warning: this compiler does not support x86 ('-arch' option ignored) gcc-13: error: unrecognized command-line option '-iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders' gcc-13: error: unrecognized command-line option '-iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers' failure. removing: _configtest.c _configtest.o building 'vt' dylib library /opt/homebrew/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -c src/lib-pmpi/vt.c -o build/temp.macosx-10.9-universal2-cpython-39/src/lib-pmpi/vt.o gcc-13: warning: this compiler does not support x86 ('-arch' option ignored) gcc-13: error: unrecognized command-line option '-iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders' gcc-13: error: unrecognized command-line option '-iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers' warning: build_clib: command '/opt/homebrew/bin/mpicc' failed with exit code 1

warning: build_clib: building optional library "vt" failed

checking for library 'vt-mpi' ... /opt/homebrew/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -c _configtest.c -o _configtest.o gcc-13: warning: this compiler does not support x86 ('-arch' option ignored) gcc-13: error: unrecognized command-line option '-iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders' gcc-13: error: unrecognized command-line option '-iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers' failure. removing: _configtest.c _configtest.o checking for library 'vt.mpi' ... /opt/homebrew/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -c _configtest.c -o _configtest.o gcc-13: warning: this compiler does not support x86 ('-arch' option ignored) gcc-13: error: unrecognized command-line option '-iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders' gcc-13: error: unrecognized command-line option '-iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers' failure. removing: _configtest.c _configtest.o building 'vt-mpi' dylib library /opt/homebrew/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -c src/lib-pmpi/vt-mpi.c -o build/temp.macosx-10.9-universal2-cpython-39/src/lib-pmpi/vt-mpi.o gcc-13: warning: this compiler does not support x86 ('-arch' option ignored) gcc-13: error: unrecognized command-line option '-iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders' gcc-13: error: unrecognized command-line option '-iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers' warning: build_clib: command '/opt/homebrew/bin/mpicc' failed with exit code 1

warning: build_clib: building optional library "vt-mpi" failed

checking for library 'vt-hyb' ... /opt/homebrew/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -c _configtest.c -o _configtest.o gcc-13: warning: this compiler does not support x86 ('-arch' option ignored) gcc-13: error: unrecognized command-line option '-iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders' gcc-13: error: unrecognized command-line option '-iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers' failure. removing: _configtest.c _configtest.o checking for library 'vt.ompi' ... /opt/homebrew/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -c _configtest.c -o _configtest.o gcc-13: warning: this compiler does not support x86 ('-arch' option ignored) gcc-13: error: unrecognized command-line option '-iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders' gcc-13: error: unrecognized command-line option '-iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers' failure. removing: _configtest.c _configtest.o building 'vt-hyb' dylib library /opt/homebrew/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -c src/lib-pmpi/vt-hyb.c -o build/temp.macosx-10.9-universal2-cpython-39/src/lib-pmpi/vt-hyb.o gcc-13: warning: this compiler does not support x86 ('-arch' option ignored) gcc-13: error: unrecognized command-line option '-iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders' gcc-13: error: unrecognized command-line option '-iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers' warning: build_clib: command '/opt/homebrew/bin/mpicc' failed with exit code 1

warning: build_clib: building optional library "vt-hyb" failed

running build_ext MPI configuration: [mpi] from 'mpi.cfg' MPI C compiler: /opt/homebrew/bin/mpicc MPI C++ compiler: /opt/homebrew/bin/mpicxx MPI F compiler: /opt/homebrew/bin/mpifort MPI F90 compiler: /opt/homebrew/bin/mpif90 MPI F77 compiler: /opt/homebrew/bin/mpif77 checking for dlopen() availability ... checking for header 'dlfcn.h' ... clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -I/Users/frederickthompson/amuse/Amuse-env/include -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/include/python3.9 -c _configtest.c -o _configtest.o success! removing: _configtest.c _configtest.o success! checking for library 'dl' ... clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -I/Users/frederickthompson/amuse/Amuse-env/include -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/include/python3.9 -c _configtest.c -o _configtest.o clang -flat_namespace -undefined suppress _configtest.o -Lbuild/temp.macosx-10.9-universal2-cpython-39 -ldl -o _configtest ld: warning: -undefined suppress is deprecated ld: warning: -undefined suppress is deprecated success! removing: _configtest.c _configtest.o _configtest checking for function 'dlopen' ... clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -I/Users/frederickthompson/amuse/Amuse-env/include -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/include/python3.9 -c _configtest.c -o _configtest.o clang _configtest.o -Lbuild/temp.macosx-10.9-universal2-cpython-39 -ldl -o _configtest success! removing: _configtest.c _configtest.o _configtest building 'mpi4py.dl' extension clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -DHAVE_DLFCN_H=1 -DHAVE_DLOPEN=1 -I/Users/frederickthompson/amuse/Amuse-env/include -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/include/python3.9 -c src/dynload.c -o build/temp.macosx-10.9-universal2-cpython-39/src/dynload.o clang -bundle -undefined dynamic_lookup -arch arm64 -arch x86_64 -Wl,-headerpad,0x1000 build/temp.macosx-10.9-universal2-cpython-39/src/dynload.o -Lbuild/temp.macosx-10.9-universal2-cpython-39 -ldl -o build/lib.macosx-10.9-universal2-cpython-39/mpi4py/dl.cpython-39-darwin.so checking for MPI compile and link ... /opt/homebrew/bin/mpicc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code -I/Users/frederickthompson/amuse/Amuse-env/include -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/include/python3.9 -c _configtest.c -o _configtest.o gcc-13: warning: this compiler does not support x86 ('-arch' option ignored) gcc-13: error: unrecognized command-line option '-iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders' gcc-13: error: unrecognized command-line option '-iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers' failure. removing: _configtest.c _configtest.o error: Cannot compile MPI programs. Check your configuration!!! error: subprocess-exited-with-error

× Building wheel for mpi4py (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip. full command: /Users/frederickthompson/amuse/Amuse-env/bin/python3 /Users/frederickthompson/amuse/Amuse-env/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /var/folders/ft/b5bx87sx2gn8sq403f33mvkw0000gq/T/tmp75mn9tm6 cwd: /private/var/folders/ft/b5bx87sx2gn8sq403f33mvkw0000gq/T/pip-install-0jzh8f15/mpi4py_27b255962f384d069e6b74e52d4550f6 Building wheel for mpi4py (pyproject.toml) ... error ERROR: Failed building wheel for mpi4py Failed to build mpi4py ERROR: Could not build wheels for mpi4py, which is required to install pyproject.toml-based projects

[LourensVeen]

A web search suggests that this -iwithsysroot option is Clang specific, and is not understood by GCC. So it seems that mpi4py expects to be building with Clang, probably XCode. It seems that mpicc forwards to Homebrew's GCC though, which then errors out on that error.

The package you have installed is called openmpi-gcc12-fortran, so that makes sense. I guess you could check if there's an openmpi-clang package, but the problem with that could be that that would then give problems installing AMUSE, which I don't think works with clang (Steven?).

[fredt00]

Yes, previously I was using the default Mac compiler which is clang and this caused the installation of AMUSE to fail later on, so I'm not sure using openmpi with clang is a good idea. In that case, I'm not sure what to try next so any advice would be appreciated!

[LourensVeen]

Ah, it looks like there's an mpi4py package in Homebrew. Could you try installing that, instead of using pip?

[rieder]

mpi with clang will give errors for AMUSE, so that should definitely be avoided. Indeed it looks like mpi4py is doing a weird mix of clang and gcc, that will cause errors. Not sure yet of the origin.

If mpi4py fails with pip, I doubt the rest of AMUSE is going to succeed even if you would install mpi4py via Homebrew...

[fredt00]

Global installation of mpi4py with Homebrew was successful, but I'm not sure how to get it working in my venv and to install the rest of AMUSE with it.

This website (https://mpi4py.readthedocs.io/en/latest/install.html) implies that you can only install mpi4py with pip if you are using mpich, so I've unlinked openmpi and am using the above Homebrew instructions with mpich instead. I will update you once it's finished installing.

[rieder]

No, the website says that if you use mpich, you can only install mpi4py from source. Openmpi is fine (and slightly recommended over mpich).

[fredt00]

Ahh ok, well the installation of mpich failed anyway so I'll go back to using openmpi.

[LourensVeen]

mpi4py only ships binaries for windows, if you're on a Mac or on Linux, pip will build it from source anyway.

The documentation linked above says that it uses setuptools by default, so I wonder now if it's maybe setuptools that generates the compile command, while blindly assuming that it's dealing with XCode and therefore clang. The relevant setuptools documentation seems to be here: https://setuptools.pypa.io/en/latest/userguide/ext_modules.html

Could it be that this -isysroot comes from Python's sysconfig? What does python3 -c 'import sysconfig ; print(sysconfig.get_config_vars("CFLAGS"))' show?

If that shows the -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers part, then maybe the problem is that you're using the system's built-in Python, and that it is set up assuming that the compiler is XCode Clang, but you have OpenMPI installed via Homebrew with its mpicc set to forward to Homebrew's GCC. So setuptools ends up running Homebrew GCC with Apple Clang options, and we get the error.

If that hypothesis is correct, then maybe installing Python through Homebrew will help? There's a reasonable chance that Homebrew's Python is compiled with Homebrew's GCC, or at least is compatible with it.

So you'd install python and openmpi using Homebrew, then verify that the python3 command really points to Homebrew's Python (which python3 should print a path in /opt/homebrew), and then you can use python3 -m venv /path/to/venv to make a virtualenv and python3 -m pip install mpi4py to install mpi4py in it. (We're calling venv and pip through the python3 command here to ensure we don't get the system ones after all.)

Then of course we still have to get the other dependencies installed and AMUSE working, but one thing at a time.

[rieder]

Ah. Yes, installing Python via homebrew/macports is probably necessary. This was missing from my instructions above.

[fredt00]

Indeed, just as you predicted:

(Amuse-env) frederickthompson@asmaclap75 amuse % python3 -c 'import sysconfig ; print(sysconfig.get_config_vars("CFLAGS"))' ['-Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers -arch arm64 -arch x86_64 -Werror=implicit-function-declaration -Wno-error=unreachable-code']

So I reinstalled Python with Homebrew and everything worked almost perfectly! I managed to install all community packages apart from amuse-fi and amuse-simplex:

Successfully built amuse-athena amuse-bhtree amuse-bse amuse-capreole amuse-evtwin amuse-fastkick amuse-fractalcluster amuse-gadget2 amuse-galactics amuse-galaxia amuse-halogen amuse-hermite amuse-hop amuse-huayno amuse-kepler amuse-kepler-orbiters amuse-mameclot amuse-mercury amuse-mmams amuse-ph4 amuse-seba amuse-secularmultiple amuse-smalln amuse-sphray amuse-sse amuse-twobody mpi4py Failed to build amuse-fi amuse-simplex ERROR: Could not build wheels for amuse-fi, amuse-simplex, which is required to install pyproject.toml-based projects

I have no idea what could be causing this, but I don't know if I'll be using these two packages anyway.

Thank you so much for all your help so far :)

[rieder]

Fi depends on fftw, simplex on hdf5. It could be that these packages were not found at the time you installed amuse-framework.