scipy-openblas64 not installable on macos_arm64

[andyfaff]

Thanks for the awesome work creating the OpenBLAS wheel. Super excited to get this working for scipy. Unfortunately something seems to be wrong with the macos_arm64 wheel:

(dev3) 192-168-1-107:scipy andrew$ pip install scipy-openblas64
ERROR: Could not find a version that satisfies the requirement scipy-openblas64 (from versions: none)

So I tried downloading the wheel for a direct install:

(dev3) 192-168-1-107:Downloads andrew$ pip install scipy_openblas64-0.3.23.293-py3-none-macosx_10_9_arm64.whl 
ERROR: scipy_openblas64-0.3.23.293-py3-none-macosx_10_9_arm64.whl is not a supported wheel on this platform.

I'm guessing that there's something wrong with the wheelname, or the metadata within the wheel?

I'm on an M1 Mac, 13.5.2, with python3.10.11.

[andyfaff]

Sidenote, if I unpack the wheel:

(dev3) 192-168-1-107:lib andrew$ otool -L libopenblas_python.so 
libopenblas_python.so:
    /opt/arm64-builds/lib/libopenblas64_.0.dylib (compatibility version 0.0.0, current version 0.0.0)
    @loader_path/../.dylibs/libgfortran.5.dylib (compatibility version 6.0.0, current version 6.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.0.0)

I'm not sure if the libopenblas_python.so is supposed to contain the actual BLAS library. However, that .so wants to load /opt/arm64-builds/lib/libopenblas64_.0.dylib. That's a hard coded link to a file that won't be present on most people's computers.

[andyfaff]

I also tried installing scipy-openblas32, in case that's the version is supposed to be using anyway), but can't pip install either.

[mattip]

Thanks for testing.

I thought I fixed the arm64 platform name, it should not be macosx_10_9_arm64 rather macosx_11_0_arm64. I must have downloaded the wrong archive.

I'm not sure if the libopenblas_python.so is supposed to contain the actual BLAS library

Yes, it is supposed to be the actual library. Checking.

[andyfaff]

Yes, it is supposed to be the actual library.

At the moment openblas.pc is pointing to a libdir of /opt/arm64-builds/lib (similar for includedir), is that usable by e.g. meson, or does meson rely on the library being in that location?

[mattip]

openblas.pc is pointing to a libdir

No that is wrong. Something seems off with the macosx wheel contents.

[mattip]

I think there is a misunderstanding. The wheel mistakenly includes a random openblas.pc file. Don't use that (I should remove it). You should do

pip install scipy-openblas32
target = <dir>
with open(target/openblas.pc, "wt", encoding="utf8") as fid:
   fid.write(scipy_openblas32.get_pgk_config())
export PKG_CONFIG_DIR=target
scipy_openblas32.write__distributor_init("numpy")
pip install . 

But better is to use one of the methods in the top of the PR

As for $ otool -L libopenblas_python.so, I think it is confused. The wheel works. I can run

% python -m scipy_openblas64
OpenBLAS using 'OpenBLAS 0.3.23.dev  USE64BITINT DYNAMIC_ARCH NO_AFFINITY armv8 MAX_THREADS=64'

[andyfaff]

Kind of unrelated, but what is the eventual aim of scipy/numpy w.r.t the openblas wheel? It is going to become a runtime dependency, or is it going to still be bundled into the wheels?

[mattip]

I think if we can make it work a runtime dependency would be nice. Then you could update OpenBLAS from a wheel independently of NumPy/SciPy, and the wheel size would shrink. On the other hand, then we enter "version roulette" where version-based issues might be harder to diagnose.

[mattip]

Closing

• the wheel can be installed on all platforms
• the direction right now is that the wheel will be used to build NumPy/SciPy, but that the shared objects will still be bundled into the PyPI wheel.