tttapa / py-build-cmake

Modern, PEP 517 compliant build backend for creating Python packages with extensions built using CMake.
https://pypi.org/project/py-build-cmake
MIT License
45 stars 7 forks source link

Minimum cmake version #6

Open cimes-isi opened 2 years ago

cimes-isi commented 2 years ago

In the process of testing #5, I ran into a separate issue with cmake version requirements. It looks like py-build-cmake itself requires at least CMake >= 3.15, if not newer. I identified this lower bound is based on support for the cmake --install CLI option (see CMake 3.15 Release Notes).

Logically: let py-build-cmake's min version be A (e.g., 3.15), the user's declared minimum version be B (None --> 0.0), and the existing system cmake version be C (None --> 0.0). If D = max(A, B), then it seems you should add the cmake >= D Python package dependency if D > C.

FWIW, I use cmake --build <dir> --target install to install on older cmake versions, but I'm not sure how cleanly that maps to your approach since it forces a "build" step as part of the install process.

tttapa commented 2 years ago

I did some tests and it looks like the oldest version of CMake where FindPython3 works reliably is 3.17. In 3.16 it silently selects the wrong Python interpreter and libraries (e.g. when invoking py-build-cmake using python3.10, CMake will ignore the paths set by py-build-cmake and link to python3.8 instead, which is very bad).

On the one hand, I don't want to constrain projects that don't need it by requiring a newer version, but on the other hand, I expect it to be a common use case to find Python in CMake when building Python packages, and the results are pretty bad if CMake somehow selects a different Python installation than the one you're using to build the package.

I feel like silently installing a newer version of CMake is better than silently generating a broken package because of an old version. As a middle ground, I would propose using 3.15 as the hard minimum version for py-build-cmake itself, and making the default version in the config file 3.17 so FindPython3 works correctly, even if the package author didn't explicitly specify it.
If you don't need to find Python for your package, you can still specify 3.15 in the config file.

cimes-isi commented 2 years ago

That's an interesting problem. I've rolled it around my head a bit and I have a number of questions that I'd want answers to before I'd feel confident supporting a particular solution. I propose that we fork that problem to a new Github issue where we can discuss it further.

In the meantime we can still solve the cmake CLI compatibility problem in front of us, which I believe is both well-defined and has a clean solution (something like I formulated above).

cimes-isi commented 2 years ago

It looks like c9a3950 should correctly resolve this. Thanks! Feel free to close this issue unless you want to continue related discussions here.

W.r.t. FindPython3, the Python build dependency on CMake (via py-build-cmake) creates a circular dependency when the native build in turn has a Python dependency. The core of the problem is that there's nothing you can do to ensure future py-build-cmake/CMake compatibility based on FindPython3 behavior unless CMake's FindPython3 guarantees forward compatibility with future Python releases, e.g., through a contract with Python installation specs (which AFAIK doesn't exist). For example, if you declare a CMake>=A.B.C dependency because FindPython3 currently works in CMake versions >=A.B.C, those FindPython3 modules still might break with a future Python release, then any existing Python build that uses py-build-cmake>=X.Y.Z and requires FindPython3 may break even though nothing here has changed (or logically needs to).

For now, it seems the simplest thing to not make any formal claims of FindPython3 compatibility and make this the user's problem, which I'd argue it is. Just document this behavior and the concerns clearly. Then if a user's project breaks in a newer Python release, they must upgrade their project's minimum CMake version to one that is compatible with that Python version. One idea to try and head off future issues to talk with the Kitware devs, e.g., to ensure that scenarios akin to a py-build-cmake build environment are included in their test suite.