search

aburrell / apexpy

A Python wrapper for Apex coordinates
MIT License
34 stars 25 forks source link

BUG: open apexypy on macOS silicon #129

Open aldebaran1 opened 7 months ago

aldebaran1 commented 7 months ago

apexpy returns an error at import: import apexpy ERROR:

fortranapex module could not be imported. apexpy probably won't work. Failed with error: cannot import name 'fortranapex' from partially initialized module 'apexpy' (most likely due to a circular import)

NOTE: fortranapex compiles normally from /frotranapex/ using make.

Using : Python 3.11.5 (main, Sep 11 2023, 08:31:25) [Clang 14.0.6 ] on darwin

This must be a problem similar to https://github.com/aburrell/apexpy/issues/16 Likely the problem with linking to the correct compiler/path in setup.cfg

aldebaran1 commented 7 months ago

PS: The problem occurs at the import. It installs normally with pip install apexpy and/or manually from git repo and pip install -e .

aburrell commented 7 months ago

I recommend following the instructions here to ensure you're using compatible compilers. This is usually the issue with macs: https://apexpy.readthedocs.io/en/latest/installation.html#specifying-compilers

aldebaran1 commented 7 months ago

Unfortunately, this doesn't work. Even if I build the package using meson and ninja (successfully), an import apexpy returns the same issue highlighted in the first message.

aburrell commented 7 months ago

Can you show me which compilers your system is using? This should be the top level output from meson.

aldebaran1 commented 7 months ago

meson setup build

The Meson build system
Version: 1.3.1
Source dir: /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy
Build dir: /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build
Build type: native build
Project name: apexpy
Project version: 2.0.1
C compiler for the host machine: cc (clang 15.0.0 "Apple clang version 15.0.0 (clang-1500.1.0.2.5)")
C linker for the host machine: cc ld64 1022.1
Host machine cpu family: aarch64
Host machine cpu: aarch64
Fortran compiler for the host machine: gfortran (gcc 13.2.0 "GNU Fortran (Homebrew GCC 13.2.0) 13.2.0")
Fortran linker for the host machine: gfortran ld64 1022.1
Compiler for C supports arguments -Wno-unused-but-set-variable: YES 
Library m found: YES
Program python3 found: YES (/Users/mraks1/miniconda3/bin/python)
Found pkg-config: YES (/usr/local/bin/pkg-config) 0.29.2
Run-time dependency python found: YES 3.11
Message: /Users/mraks1/miniconda3/bin/python
Message: /usr/local/lib/python3.11/site-packages/
Check usable header "Python.h" with dependency python-3.11: YES 
Build targets in project: 3

Found ninja-1.11.1.git.kitware.jobserver-1 at /Users/mraks1/miniconda3/bin/ninja

ninja -j 2 -C build executes, but I get the following warnings:

ld: warning: ignoring file '/Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build/fortranapex.cpython-311-darwin.so.p/fortranapex_apex.f90.o': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build/fortranapex.cpython-311-darwin.so.p/fortranapex_magfld.f90.o': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build/fortranapex.cpython-311-darwin.so.p/meson-generated_.._fortranapex-f2pywrappers2.f90.o': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build/fortranapex.cpython-311-darwin.so.p/fortranapex_makeapexsh.f90.o': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build/fortranapex.cpython-311-darwin.so.p/fortranapex_igrf.f90.o': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build/fortranapex.cpython-311-darwin.so.p/fortranapex_apexsh.f90.o': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build/fortranapex.cpython-311-darwin.so.p/fortranapex_checkapexsh.f90.o': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/usr/local/Cellar/gcc/13.2.0/lib/gcc/current/libgfortran.5.dylib': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/usr/local/Cellar/gcc/13.2.0/lib/gcc/current/libquadmath.0.dylib': found architecture 'x86_64', required architecture 'arm64'

meson install:

[9/9] Linking target fortranapex.cpython-311-darwin.so
ld: warning: ignoring file '/Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build/fortranapex.cpython-311-darwin.so.p/fortranapex_magfld.f90.o': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build/fortranapex.cpython-311-darwin.so.p/fortranapex_apex.f90.o': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build/fortranapex.cpython-311-darwin.so.p/fortranapex_makeapexsh.f90.o': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build/fortranapex.cpython-311-darwin.so.p/meson-generated_.._fortranapex-f2pywrappers2.f90.o': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build/fortranapex.cpython-311-darwin.so.p/fortranapex_apexsh.f90.o': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build/fortranapex.cpython-311-darwin.so.p/fortranapex_checkapexsh.f90.o': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build/fortranapex.cpython-311-darwin.so.p/fortranapex_igrf.f90.o': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/usr/local/Cellar/gcc/13.2.0/lib/gcc/current/libquadmath.0.dylib': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/usr/local/Cellar/gcc/13.2.0/lib/gcc/current/libgfortran.5.dylib': found architecture 'x86_64', required architecture 'arm64'
Installing fortranapex.cpython-311-darwin.so to /usr/local/lib/python3.11/site-packages/apexpy
Installing /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/apexpy/__init__.py to /usr/local/lib/python3.11/site-packages/apexpy
Installing /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/apexpy/__main__.py to /usr/local/lib/python3.11/site-packages/apexpy
Installing /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/apexpy/apex.py to /usr/local/lib/python3.11/site-packages/apexpy
Installing /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/apexpy/apexsh.dat to /usr/local/lib/python3.11/site-packages/apexpy
Installing /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/apexpy/helpers.py to /usr/local/lib/python3.11/site-packages/apexpy
Installing /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/apexpy/igrf13coeffs.txt to /usr/local/lib/python3.11/site-packages/apexpy
Running custom install script '/Users/mraks1/miniconda3/bin/python /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build/meson-private/pycompile.py python-3.11-installed.json 0'
Compiling '/usr/local/lib/python3.11/site-packages/apexpy/__init__.py'...
Compiling '/usr/local/lib/python3.11/site-packages/apexpy/__main__.py'...
Compiling '/usr/local/lib/python3.11/site-packages/apexpy/apex.py'...
Compiling '/usr/local/lib/python3.11/site-packages/apexpy/helpers.py'...
aburrell commented 7 months ago

Here's the problem:

C compiler for the host machine: cc (clang 15.0.0 "Apple clang version 15.0.0 (clang-1500.1.0.2.5)") C linker for the host machine: cc ld64 1022.1

You need to specify the C compiler to match the fortran compiler, both need to be GCC.

aldebaran1 commented 7 months ago

Ok, I did, but without success.

meson setup build

The Meson build system
Version: 1.3.1
Source dir: /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy
Build dir: /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build
Build type: native build
Project name: apexpy
Project version: 2.0.1
C compiler for the host machine: /usr/local/bin/gcc-13 (gcc 13.2.0 "gcc-13 (Homebrew GCC 13.2.0) 13.2.0")
C linker for the host machine: /usr/local/bin/gcc-13 ld64 1022.1
Host machine cpu family: aarch64
Host machine cpu: aarch64
Fortran compiler for the host machine: /usr/local/bin/gfortran (gcc 13.2.0 "GNU Fortran (Homebrew GCC 13.2.0) 13.2.0")
Fortran linker for the host machine: /usr/local/bin/gfortran ld64 1022.1
Compiler for C supports arguments -Wno-unused-but-set-variable: YES 
Library m found: YES
Program python3 found: YES (/Users/mraks1/miniconda3/bin/python)
Found pkg-config: YES (/usr/local/bin/pkg-config) 0.29.2
Run-time dependency python found: YES 3.11
Message: /Users/mraks1/miniconda3/bin/python
Message: /usr/local/lib/python3.11/site-packages/
Check usable header "Python.h" with dependency python-3.11: YES 
Build targets in project: 3

Found ninja-1.11.1.git.kitware.jobserver-1 at /Users/mraks1/miniconda3/bin/ninja

meson install:

Installing fortranapex.cpython-311-darwin.so to /usr/local/lib/python3.11/site-packages/apexpy
Installing /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/apexpy/__init__.py to /usr/local/lib/python3.11/site-packages/apexpy
Installing /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/apexpy/__main__.py to /usr/local/lib/python3.11/site-packages/apexpy
Installing /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/apexpy/apex.py to /usr/local/lib/python3.11/site-packages/apexpy
Installing /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/apexpy/apexsh.dat to /usr/local/lib/python3.11/site-packages/apexpy
Installing /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/apexpy/helpers.py to /usr/local/lib/python3.11/site-packages/apexpy
Installing /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/apexpy/igrf13coeffs.txt to /usr/local/lib/python3.11/site-packages/apexpy
Running custom install script '/Users/mraks1/miniconda3/bin/python /Users/mraks1/Library/CloudStorage/GoogleDrive-sebastijan.mrak@gmail.com/My Drive/BU/software/apexpy/build/meson-private/pycompile.py python-3.11-installed.json 0'
Compiling '/usr/local/lib/python3.11/site-packages/apexpy/__init__.py'...
Compiling '/usr/local/lib/python3.11/site-packages/apexpy/__main__.py'...
Compiling '/usr/local/lib/python3.11/site-packages/apexpy/apex.py'...
Compiling '/usr/local/lib/python3.11/site-packages/apexpy/helpers.py'...

However, python doesn't find apexpy now. It looks meson installs it in usr/local/lib/python/.... instead of in /miniconda3/lib/python.... Do you have a suggestion on how to specify the correct path for installation? It looks like meson setup finds the correct path to /miniconda3/...

aburrell commented 3 months ago

Ok, so I don't know how to change where pip or meson installs site packages. You can add that path to your python path and that is one way forward. However, I think I have the right tricks now to let you use pip directly via:

CC=/usr/local/bin/gcc-13 FC=/usr/local/bin/gfortran pip install apexpy

On a recent M1 Mac with Anaconda Python, I ran into the following additional issues:

1) Needed to ensure pkg-config was installed: conda install pkg-config 2) Needed x86 architecture gcc: had to install brew via arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" in order to install gfortran with the correct arc. via the normal brew install gcc@13 or similar

This allowed the conda pip to install apexpy using the specified compilers and run successfully. If number (2) is an issue, you'll see apexpy look like it installs successfully. Then, when you import you'll get an error message that it can't read the fortran file because of the architecture.