mpicc not found during uw3 installation in mac arm

[gthyagi]

Hi,

Can someone double-check whether mpi4py.get_config() is returning the correct path or if it's empty? On a Mac with an ARM chip, this command outputs an empty result, leading to the mpicc not found error.

https://github.com/underworldcode/underworld3/blob/275e624ab979f821494592b45a06b582985e5e63/setup.py#L22

Thanks.

[bknight1]

works fine for me. Does which mpirun show the correct path in terminal for you?

[gthyagi]

problem is mpi4py is not detecting my existing mpicc

underworld3 git:(development) which mpicc
/Users/tgol0006/manual_install_pkg/openmpi-4.1.6/bin/mpicc

[gthyagi]

These installation instructions I am using https://github.com/gthyagi/tools_install_guide/blob/main/uw3_mac_install_arm.sh

[bknight1]

Looks like it's added to your path correctly. You can try passing it to mpi4py directly during install:

https://mpi4py.readthedocs.io/en/latest/develop.html

you can try: mpiexec -n 5 python -m mpi4py.bench helloworld to see if mpi4py is setup correctly

[gthyagi]

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/bench.py", line 587, in <module>
    main()
  File "/Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/bench.py", line 568, in main
    from . import MPI
ImportError: dlopen(/Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/MPI.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libmpi.12.dylib
  Referenced from: <6D1BC676-20B5-3925-B69A-72A7062C4617> /Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/MPI.cpython-311-darwin.so
  Reason: tried: '/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/opt/homebrew/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/lib/libmpi.12.dylib' (no such file), '/opt/homebrew/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/lib/libmpi.12.dylib' (no such file)
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/bench.py", line 587, in <module>
    main()
  File "/Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/bench.py", line 568, in main
    from . import MPI
ImportError: dlopen(/Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/MPI.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libmpi.12.dylib
  Referenced from: <6D1BC676-20B5-3925-B69A-72A7062C4617> /Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/MPI.cpython-311-darwin.so
  Reason: tried: '/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/opt/homebrew/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/lib/libmpi.12.dylib' (no such file), '/opt/homebrew/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/lib/libmpi.12.dylib' (no such file)
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/bench.py", line 587, in <module>
    main()
  File "/Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/bench.py", line 568, in main
    from . import MPI
ImportError: dlopen(/Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/MPI.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libmpi.12.dylib
  Referenced from: <6D1BC676-20B5-3925-B69A-72A7062C4617> /Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/MPI.cpython-311-darwin.so
  Reason: tried: '/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/opt/homebrew/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/lib/libmpi.12.dylib' (no such file), '/opt/homebrew/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/lib/libmpi.12.dylib' (no such file)
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/bench.py", line 587, in <module>
    main()
  File "/Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/bench.py", line 568, in main
    from . import MPI
ImportError: dlopen(/Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/MPI.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libmpi.12.dylib
  Referenced from: <6D1BC676-20B5-3925-B69A-72A7062C4617> /Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/MPI.cpython-311-darwin.so
  Reason: tried: '/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/opt/homebrew/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/lib/libmpi.12.dylib' (no such file), '/opt/homebrew/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/lib/libmpi.12.dylib' (no such file)
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/bench.py", line 587, in <module>
    main()
  File "/Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/bench.py", line 568, in main
    from . import MPI
ImportError: dlopen(/Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/MPI.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libmpi.12.dylib
  Referenced from: <6D1BC676-20B5-3925-B69A-72A7062C4617> /Users/tgol0006/manual_install_pkg/venv_uw3_git_5924/venv_uw3/lib/python3.11/site-packages/mpi4py/MPI.cpython-311-darwin.so
  Reason: tried: '/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/tgol0006/micromamba/envs/UW3_16_8_24/lib/libmpi.12.dylib' (no such file), '/opt/homebrew/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/lib/libmpi.12.dylib' (no such file), '/opt/homebrew/lib/libmpi.12.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/lib/libmpi.12.dylib' (no such file)
--------------------------------------------------------------------------
mpiexec detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[19673,1],0]
  Exit code:    1

[bknight1]

Okay it looks like it's searching through homebrew and micromamba for the dylib files, whereas its installed somewhere in '/Users/tgol0006/manual_install_pkg/openmpi-4.1.6'. You might be able to add the directory during install by adding something like: --mpicc=/path/to/mpicc

This thread may help, which seems to have similar issues to what you're having: https://groups.google.com/g/mpi4py/c/xpPKcOO-H4k

here's the workaround: https://groups.google.com/g/mpi4py/c/xpPKcOO-H4k/m/8oBGf9lfAgAJ

[gthyagi]

okay, I am puzzled why my mpi4py does not recongize existing openmpi which comes .zshrc. Even export CC=$(which mpicc) also does the job.

[bknight1]

Was it installed through conda/mamba? That is what may be causing issues with your native install

[gthyagi]

No, openmpi was installed separately, as system wide install instead of env specific.

[gthyagi]

Was it installed through conda/mamba? That is what may be causing issues with your native install

that makes sense. seems google groups has indepth discussion related to this. thanks

[gthyagi]

I am thinking of adding this code to setup.py

if os.environ.get("CC") is None:
    import warnings
    warnings.warn("CC environment variable not set. Attempting to use mpi4py's compiler configuration.")

    # Get CC from mpi4py
    import mpi4py
    conf = mpi4py.get_config()

    # Check if "mpicc" is in the mpi4py configuration
    if "mpicc" in conf:
        os.environ["CC"] = conf["mpicc"]
    else:
        # Fallback: manually set CC to the path of the mpicc compiler
        mpicc_path = os.popen('which mpicc').read().strip()
        if mpicc_path:
            os.environ["CC"] = mpicc_path
        else:
            raise KeyError("'mpicc' not found in mpi4py configuration, and 'mpicc' not found in system path. "
                           "Ensure MPI is installed and 'mpicc' is available.")

[gthyagi]

this is resolved in #247

[bknight1]

After mpi4py/CC config was removed in #247 I had trouble installing/updating UW as I had not set the CC environment variable in my terminal.

/Users/benknight/software/petsc-v3.21.2/include/petscsys.h:65:12: fatal error: 'mpi.h' file not found
  #include <mpi.h>

This can be set through: export CC=$(which mpicc) as highlighted by @gthyagi. This should be done before any updates/installs.

[bknight1]

I would probably add something like this back in:

mpicc_path = os.popen('which mpicc').read().strip()
os.environ["CC"] = mpicc_path

[gthyagi]

Yes, please modify the current setup.py file. It consistently fails with an error stating that mpi.h is not found on macOS.