Wrong cython3 found during setup

[mattip]

Describe the bug I am not sure this is a problem with meson or cython. TL;DR: meson looks for a binary named cython3, but cython inside a virtual environment does not link bin/cython to bin/cython3

To Reproduce I am trying to convert the NumpPy numpy/random/_examples (used to test building c-extensions using the random cython pyx files) from setup.py to meson. So far I have the meson.bulid file below. When I do meson setup in a virtualenv, it looks for cython3 not cython , and finds cython3 in /usr/bin/cython3 rather than cython in <venv>/bin/cython . That is the wrong version of cython so the build fails. Is the problem that meson is looking for cython3, or that Cython doesn't install a cython3 binary?

project('random-build-examples', 'c', 'cpp', 'cython')

# https://mesonbuild.com/Python-module.html
py_mod = import('python')
py3 = py_mod.find_installation(pure: false)
py3_dep = py3.dependency()

py_mod = import('python')
py = py_mod.find_installation(pure: false)
cc = meson.get_compiler('c')
cy = meson.get_compiler('cython')

if not cy.version().version_compare('>=0.29.35')
  error('tests requires Cython >= 0.29.35')
endif

_incdir_numpy_abs = run_command(py3,
    ['-c', 'import os; os.chdir(".."); import numpy; print(numpy.get_include())'],
    check: true
  ).stdout().strip()

npymath_path = _incdir_numpy_abs / '..' / 'lib'
npyrandom_path = _incdir_numpy_abs / '..' / '..' / 'random' / 'lib'
npymath_lib = cc.find_library('npymath', dirs: npymath_path)
npyrandom_lib = cc.find_library('npyrandom', dirs: npyrandom_path)

py.extension_module(
    'extending_distributions',
    'extending_distributions.pyx',
    install: true,
    include_directories: [_incdir_numpy_abs],
    dependencies: [npyrandom_lib, npymath_lib],
)

Expected behavior When running meson setup I expect to see

Cython compiler for the host machine: cython3 (cython 0.29.36)

but see

Cython compiler for the host machine: cython3 (cython 0.29.28)

system parameters

• Is this a cross build or just a plain native build (for the same computer)? Native
• what operating system (e.g. MacOS Catalina, Windows 10, CentOS 8.0, Ubuntu 18.04, etc.) Ubuntu 22.04
• what Python version are you using e.g. 3.8.0 3.11
• what meson --version 1.1.1
• what ninja --version if it's a Ninja build NA

[eli-schwartz]

https://github.com/mesonbuild/meson/blob/ecbba0c45b05b53563b23b84191a0acccdfcc291/mesonbuild/compilers/detect.py#L83

What should happen here is that we first look for cython, and if that still isn't found, we try to look again for cython3. The latter is flat-out incorrect, wrong, and should not exist and should not be looked up...

... but we do it anyway because Debian provides a /usr/bin/cython that is written in python2, and a /usr/bin/cython3 that is written in python3. Their rationale is that the "3" in "cython3" is the version of python whose interpreter it uses.

There is a debian discussion to resolve this, see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=686045#15

[eli-schwartz]

The question is, why doesn't meson find the preferred cython binary from inside the virtualenv? Did you use pip / python -m build, or . venv/bin/activate followed by manually invoking meson? The only thing I could think of off the top of my head is that somehow the virtualenv version of cython is not in $PATH at the point where meson is getting run.

[mattip]

I activated the virtualenv.

Strange, when I delete the build directory and start again, the correct version is picked up. It must be some cached value messing things up while I put the meson.build directory together, and somehow meson setup --reconfigure is getting confused. I will close as "not reproducible"

[eli-schwartz]

Ah, caching issues could be why I suppose. Thanks for the update.