`ninja==1.11.1.2` build fails when built with `--no-binary=ninja` (or `--no-binary=:all:`)

[TimotheusBachinger]

Since ninja==1.11.1.2, we cannot built numpy anymore from source as it will fail with:

LookupError: https://files.pythonhosted.org/packages/b4/49/4f1a79f99f4c3eb5d22f943bba14832923bb44423254d5089d38a9f6da63/ninja-1.11.1.2.tar.gz (from https://pypi.org/simple/ninja/) (requires-python:>=3.7) is already being built: ninja>=1.8.2 from https://files.pythonhosted.org/packages/b4/49/4f1a79f99f4c3eb5d22f943bba14832923bb44423254d5089d38a9f6da63/ninja-1.11.1.2.tar.gz

numpy does not pin ninja in its build dependencies, which explains why the build is now failing: https://github.com/numpy/numpy/blob/b1e6ccd14b173b922f5d6a11bb252f4141dc42cd/requirements/build_requirements.txt#L3

Do you have any idea what causes this?

[johningve]

I'm also having issues with this new version. It seems like ninja is calling itself recursively:

[mayeut]

@TimotheusBachinger, it seems like the bootstrapping issue mentioned in https://github.com/scikit-build/cmake-python-distributions/issues/503#issuecomment-2191858783

ninja requires cmake and cmake is requiring ninja. Installing the system cmake should solve the issue.

Any reason you're not getting / using binary wheels ? (specific platform maybe ?)

@johningve, this doesn't seem like the same issue, can you please open a different issue with the reproducer please ?

[TimotheusBachinger]

@mayeut thanks for pointing into that direction - this could really have caused the issue in our case (and due to heavy caching by bazel, to use it looked like the new ninja version...)

[TimotheusBachinger]

I am re-opening the issue again as the cmake theory was a dead end for us. Our quick fix for now it to use a constrain when building from source: https://github.com/Checkmk/checkmk/commit/03164646ebcb5b7af7cf00a9c918770e4ed1cbf1

For us it is 100% the new version of ninja==1.11.1.2 which leds to the build failure.

[mayeut]

@TimotheusBachinger,

In order to better understand the problem and provide a fix:

Any reason you're not getting / using binary wheels ? Force build from source or specific platform maybe ?

[TimotheusBachinger]

@mayeut we're building for around 12 different platforms (debian, ubuntu, sles, centos, alma...) therefore we're building from source, yes (--no-binary=":all:")

[mayeut]

ok, so I have an idea where the issue is coming from in that case. I will try something after day work to reproduce and hopefully fix this. It's still a bootstrapping issue which this time should be handled by the new build system used by Ninja PyPI distribution.

[TimotheusBachinger]

@mayeut sounds good, thanks for the effort and no hurry. We're fine with the constraints workaround.

[mayeut]

Here's a quick status: The build backend switched over to scikit-build-core. scikit-build-core is aware of platforms where binary wheels are available for ninja and will insert a dependency on ninja in the build requirements if a satisfying version is not found on the system. In a --no-binary=":all:" (or just --no-binary=ninja) condition, trying to request ninja while building ninja triggers a recursive build of ninja (caught by the build frontend: ... is already being built: ninja>=1.8.2 ...).

I'll propose something a bit better than what's already there in https://github.com/scikit-build/ninja-python-distributions/pull/276 and let our resident expert review this.

[cdce8p]

I'm seeing the same issue when trying to build ninja itself. A reproducer in case that's useful

docker pull python:3.13.0
docker run -it --rm python:3.13.0 bash

pip install -v --no-binary ninja ninja==1.11.1.2

A temporary workaround is to use --no-build-isolation.

docker run -it --rm python:3.13.0 bash

pip install scikit-build-core "hatch-fancy-pypi-readme>=22.3" setuptools-scm
pip install -v --no-binary ninja --no-build-isolation ninja==1.11.1.2