Module API version issue when numpy is installed from source and numpy version is older than `oldest-supported-numpy`

[lesteve]

This is a bit of a edge case, but this would be good to have your thoughts on it, since this affects all the packages that uses oldest-supported-numpy. Maybe there is not much we can do to improve the situation.

Currently if you have Python 3.11, install numpy 1.22 from source, and install a wheel for scipy (or any package that uses oldest-supported-numpy) you will get an error at import time about ABI version mismatch.

One way to reproduce:

conda create -n testenv python=3.11 -c conda-forge -y
conda activate testenv
pip install 'numpy<1.23' scipy

python -c 'import scipy.linalg'

Output:

RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xf
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/lesteve/miniconda3/envs/test/lib/python3.11/site-packages/scipy/linalg/__init__.py", line 197, in <module>
    from ._misc import *
  File "/home/lesteve/miniconda3/envs/test/lib/python3.11/site-packages/scipy/linalg/_misc.py", line 3, in <module>
    from .blas import get_blas_funcs
  File "/home/lesteve/miniconda3/envs/test/lib/python3.11/site-packages/scipy/linalg/blas.py", line 213, in <module>
    from scipy.linalg import _fblas
ImportError: numpy.core.multiarray failed to import

In some sense this is slightly weird to do that since numpy 1.23 is the first version that officially supports Python 3.11. At the same time, the user experience is not great.

Also some distributions may package numpy 1.22 for Python 3.11, e.g. Fedora, and you will end up in a similar situation if you install a scipy wheel on top of your package manager numpy. This actually came up in the scikit-learn repo, see https://github.com/scikit-learn/scikit-learn/issues/25159 for more details.

[rgommers]

@lesteve thanks for bringing this up. There's indeed a couple of "don't do that" things here, and limitations of Python packaging:

• Explicitly installing a too-old version of numpy is unhelpful
• Fedora should not be packaging 1.22, but 1.23 for 3.11. That is worth a bug report to Fedora I'd say. It's nice that they are very quick to package new Python releases, but this is a bad idea.
• There is no upper bound on the numpy sdist/wheels; 1.22.x supports 3.8-3.10 while setup.py uses `python_requires>='3.8'. There are downsides to the upper bound as well, because some tools (IIRC Poetry) are not handling it correctly.

However, the biggest issue here is that the wheels for scipy, scikit-learn or other users of oldest-supported-numpy do not contain the correct lower bound on the runtime version of numpy that they need. This is and always has been wrong, mainly because setuptools/numpy.distutils do not support setting this correctly based on the build-time version of numpy used. The way to fix it is to implement pin_compatible so that the wheel metadata will be correct. See https://github.com/mesonbuild/meson-python/issues/29

[gst]

Hi,

I have been impacted by this as well, on aarch64, actually, I think it's maybe not the first time but I simply spotted it this time :

basically, pycuda and pycocotools are using oldest-supported-numpy in their pyproject.toml build-dependencies. and given they only provide source/tar.gz I believe this is impacting eventually lot of people.

I had to patch them with :

- "oldest-supported-numpy",
+ "numpy",

in their respective pyproject.toml to be able to build them appropriatedly or correctly with latest numpy (>= 1.20) basically.

[rgommers]

@gst that sounds like a different problem. If you want to build against the latest rather than the oldest possible version of numpy, then you should turn off build isolation (--no-build-isolation for pip, or --no-isolation --skip-dependency-check for pypa/build), rather than patch the metadata.

[gst]

ok --no-build-isolation makes the job.. if you have already the correct build dependencies installed though. hmm.

--skip-dependency-check is not really an option IMHO.

maybe this should be reported to pycuda and pycocotools repositories then, I don't know.

[ArTourter]

I agree with @gst , --skip-dependency-check should not really be an option. I maintain packages for Slackbuilds.org where we provide build scripts for slackware rather than directly installable packages and with now a few python modules starting to have oldest-supported-numpy in their requirement, it feels like removing the feature that build provides to ensure the correct packages are install is rather counter productive. I guess replacing the absolute == test with minimal required version >= would solve the problem. Having a version of numpy that is newer than the oldest possible version should still satisfy the requirements.

[rgommers]

it feels like removing the feature that build provides to ensure the correct packages are install is rather counter productive.

It's an extremely limited feature, for multiple reasons; why it cannot apply here (no static metadata for C API/ABI constraints basically) has been elaborated on before. There's nothing that can be done until pyproject.toml metadata gains support for C ABI constraints. That has been worked on, but at the current rate it's going to take a while to land.

I maintain packages for Slackbuilds.org where we provide build scripts for slackware rather than directly installable packages

There's an additional conceptual issue here. pyproject.toml metadata is primarily/only meant for building wheels from an sdist within a PyPI-focused context. That is clearly not the same as for another packaging ecosystem - whether that's a Linux distro, Homebrew, conda, or something else. That other packaging ecosystems all have their own metadata files/patches. It seems to me like you're in a similar situation, and you cannot expect pyproject.toml metadata to work unchanged. I know that's confusing, but it's a conceptual gap in Python packaging.

[rgommers]

I'm going to close this as wontfix - this isn't really actionable. The original issue with numpy 1.22 is unsupported I'd say, and user error / bad action from Fedora.

Also some distributions may package numpy 1.22 for Python 3.11, e.g. Fedora, and you will end up in a similar situation if you install a scipy wheel on top of your package manager numpy.

I'd say that is also a "don't do that situation". I understand the problem, but there is no real solution other than teaching people to not mix package managers.

SciPy doesn't use oldest-supported-numpy by the way; the problem is the same though.