0.15.0: issue on building `numpy`

[kloczek]

I'm trying to build numpy 1.26.4 in build env in which is installed meson-python and meson 1.4.0, On building numpy using pep517 based build looks like meson-python has issue with meson detection

* Getting build dependencies for wheel...
* Building wheel...

('\x1b[31m',)meson-python: error: Could not find meson version 0.63.3 or newer, found .

ERROR Backend subprocess exited when trying to invoke build_wheel

In mesonpy/init.py I see fragment responsible for printing that essage:

    required_version = _parse_version_string(version)
    meson_version = subprocess.run(cmd + ['--version'], check=False, text=True, capture_output=True).stdout
    if _parse_version_string(meson_version) < required_version:
        raise ConfigError(f'Could not find meson version {version} or newer, found {meson_version}.')

    return cmd

So looks like it tries to use meson --version command. In my case it returns:

[tkloczko@pers-jacek SPECS]$ meson --version
1.4.0

What could be wrong in this case? 🤔

[eli-schwartz]

Well, you haven't actually provided enough information to debug the issue.

What source tarball of numpy are you using, for example?

[rgommers]

Please provide the exact install command used and a full build log with complete traceback.

[dnicolodi]

My reading of the coffee grounds is that this is due to two pip issues interacting:

• meson is installed in the virtualenv where numpy is being installed and pip is building with build isolation enabled: this causes the meson executable installed in the virtualenv not to be visible to meson-python,
• therefore, meson-python asks for meson to be installed in the isolated build environment, however, this it an old pip release with a known bug resulting in executables installed in the build environment to not be visible from the build process.

Solutions: either build with build isolation disabled (pip's --no-build-isolation command line option) or upgrade pip. Upgrading pip is probably a good idea anyway. And if you keep build isolation enabled (the recommended default) you don't need to install meson or meson-python in the destination Python environment.

If my suppositions are confirmed, this is the second time in a not too long time that someone is bitten by this pip bug. We could add a warning if we detect that the build is initiated by a known-to-be-broken pip version. I'll see if there is a reliable way to do this.

[kloczek]

Please provide the exact install command used and a full build log with complete traceback.

Here it is (one line more only which I've been thinking is obvious 😋):

+ /usr/bin/python3 -sBm build -w --no-isolation
* Getting build dependencies for wheel...
* Building wheel...

('\x1b[31m',)meson-python: error: Could not find meson version 0.63.3 or newer, found .

ERROR Backend subprocess exited when trying to invoke build_wheel

So I'm using pep517 based build.

[rgommers]

meson-python version too old? You need 0.15.0

[kloczek]

meson-python version too old? You need 0.15.0

Please have look one more time on ticket title. This ticket is about 0.15.0 😋

[kloczek]

Nevertheless .. what can I do about diagnose this issue? 🤔

[dnicolodi]

My reading of the coffee grounds was not that accurate, apparently.

From the error message, it seems that running meson --version fails. Which version of NumPy are you trying to build? Which version of build are you using? If this is a recent Numpy release, the meson version installed in the system is irrelevant, as Numpy uses a vendored copy of meson. Which version of Python? How is the Python environment created? Which OS?

[dnicolodi]

By any change, do you have a $MESON environment variable set?

[rgommers]

Or an uninitialized git submodule? python vendored-meson/meson/meson.py --version in the root of the numpy repo (if you're building from a repo tag) should succeed.

This is almost certainly user error, but without all the details it's hard to diagnose.

[kloczek]

By any change, do you have a $MESON environment variable set?

Nope .. meson is on the path

[tkloczko@pers-jacek SPECS]$ command -v meson
/usr/bin/meson

Or an uninitialized git submodule? python vendored-meson/meson/meson.py --version in the root of the numpy repo (if you're building from a repo tag) should succeed.

One sec .. so you want to say that meson-python needs to use OWN COPY of the whole meson and cannot use meson installed in system resources (which is even visible as python module? 🤔 )

[tkloczko@pers-jacek SPECS]$ pip show meson
Name: meson
Version: 1.4.0
Summary: A high performance build system
Home-page: https://mesonbuild.com
Author: Jussi Pakkanen
Author-email: jpakkane@gmail.com
License: Apache License, Version 2.0
Location: /usr/lib/python3.9/site-packages
Requires:
Required-by:

Really? 🤔

[dnicolodi]

It is really not possible to find what you are doing wrong unless you provide all the required information and stop answering only selected questions and reading only parts of our answers.

Really?

No. No one said that. It is NumPy that uses a vendored copy of meson which includes some features not yet present in any meson release.

[kloczek]

Really?

No. No one said that. It is NumPy that uses a vendored copy of meson which includes some features not yet present in any meson release.

Hmm ,. but where do you see that vendored meson in numpy tree https://github.com/numpy/numpy/? 🤔

Message which I've quoted is printed by meson-python.

Did you try to reproduce reported issue? 🤔

[eli-schwartz]

In the very first reply to this ticket, I said:

Well, you haven't actually provided enough information to debug the issue.

What source tarball of numpy are you using, for example?

I'm still waiting for an answer. I know exactly what the problem is, and I knew it all along, I just want @kloczek to provide the basics of a bug report for once to prove it.

Anyone else is of course free to theorize based on incomplete information, but my methodology has a reason and you might be wise to follow it...

[dnicolodi]

Hmm ,. but where do you see that vendored meson in numpy tree https://github.com/numpy/numpy/? 🤔

https://github.com/numpy/numpy/tree/main/vendored-meson

Did you try to reproduce reported issue? 🤔

Absolutely. And there are CI jobs that build NumPy and they do not reproduce the issue. Therefore, it must be something specific to your setup. However, you refuse to provide any kind of useful detail about it. Therefore, it is impossible to help you.

[dnicolodi]

I know exactly what the problem is, and I knew it all along

You could have said. I would not have thrown away all this time trying to get more details 😄

[kloczek]

Hmm ,. but where do you see that vendored meson in numpy tree https://github.com/numpy/numpy/? 🤔

https://github.com/numpy/numpy/tree/main/vendored-meson

Holly <censored> .. nobody expects the Spanish inquisition 😵‍💫

I'm using tar ball autogenerated from git tag.

OK closing.

[eli-schwartz]

In retrospect it might have been a mistake. I was originally trying to force the issue by demanding a properly detailed bug report before continuing further, but I didn't consider that others would also try to figure out the cause based on missing details.

(@rgommers' latest comment got it as well. It's very clearly the wrong source code being used.)

[kloczek]

BTW maybe it is possible to print a bit better mesaże that it was not possible to find meson cli passed over pyproject.toml? 🤔 Really last thing which I've been anticipating was that numpy uses vendored meson😕 Sorry ..

[kloczek]

In retrospect it might have been a mistake. I was originally trying to force the issue by demanding a properly detailed bug report before continuing further, but I didn't consider that others would also try to figure out the cause based on missing details.

OK that was my pure fault. Meson as the projekt is really well maintained and only time to time has some minor issues. In that situation cheking which one meson is using build was not on my check-possible-cauises list. Vendoring meson is really odd idea/approach ..

[kloczek]

As I'm working on whole distribution it wold be in my case way better to test regular meson installed in system image (as yet another opportunity tu test it. Using below patch

--- a/pyproject.toml
+++ b/pyproject.toml
@@ -181,15 +181,9 @@
 config-settings = "setup-args=--vsenv setup-args=-Dallow-noblas=true"
 repair-wheel-command = ""

-[tool.meson-python]
-meson = 'vendored-meson/meson/meson.py'
-
 [tool.spin]
 package = 'numpy'

-[tool.spin.meson]
-cli = 'vendored-meson/meson/meson.py'
-
 [tool.spin.commands]
 "Build" = [
   ".spin/cmds.py:build",

I was able to pass meson detection but looks like meson fails

+ /usr/bin/python3 -sBm build -w --no-isolation
* Getting build dependencies for wheel...
* Building wheel...
+ meson setup /home/tkloczko/rpmbuild/BUILD/numpy-1.26.4 /home/tkloczko/rpmbuild/BUILD/numpy-1.26.4/.mesonpy-vca7x9cm -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/home/tkloczko/rpmbuild/BUILD/numpy-1.26.4/.mesonpy-vca7x9cm/meson-python-native-file.ini
The Meson build system
Version: 1.4.0
Source dir: /home/tkloczko/rpmbuild/BUILD/numpy-1.26.4
Build dir: /home/tkloczko/rpmbuild/BUILD/numpy-1.26.4/.mesonpy-vca7x9cm
Build type: native build
Project name: NumPy
Project version: 1.26.5
C compiler for the host machine: /usr/bin/gcc (gcc 14.0.1 "gcc (GCC) 14.0.1 20240316 (Red Hat 14.0.1-0)")
C linker for the host machine: /usr/bin/gcc ld.bfd 2.42.50.20240304
C++ compiler for the host machine: /usr/bin/g++ (gcc 14.0.1 "g++ (GCC) 14.0.1 20240316 (Red Hat 14.0.1-0)")
C++ linker for the host machine: /usr/bin/g++ ld.bfd 2.42.50.20240304
Cython compiler for the host machine: cython (cython 3.0.9)
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python found: YES (/usr/bin/python3)
Found pkg-config: YES (/usr/bin/pkg-config) 2.1.0
Run-time dependency python found: YES 3.9
Has header "Python.h" with dependency python-3.9: YES
Compiler for C supports arguments -fno-strict-aliasing: YES

../meson_cpu/x86/meson.build:2:15: ERROR: Module "features" does not exist

A full log can be found at /home/tkloczko/rpmbuild/BUILD/numpy-1.26.4/.mesonpy-vca7x9cm/meson-logs/meson-log.txt

ERROR Backend subprocess exited when trying to invoke build_wheel

[dnicolodi]

NumPy uses a vendored Meson because it relies on functionality that has not been yet integrated upstream.

[kloczek]

NumPy uses a vendored Meson because it relies on functionality that has not been yet integrated upstream.

Do you know is it any plan/someone is working on integrate those functionalities into regular meson? 🤔

[dnicolodi]

https://github.com/mesonbuild/meson/pull/10921 https://github.com/mesonbuild/meson/pull/11307