tttapa / py-build-cmake

Modern, PEP 517 compliant build backend for creating Python packages with extensions built using CMake.
https://pypi.org/project/py-build-cmake
MIT License
45 stars 7 forks source link

Q: Passing CMake variables on the command line #23

Open SanderVocke opened 10 months ago

SanderVocke commented 10 months ago

First of all: thanks for this very useful tool.

I was wondering whether it is a concsious decision (or result of a decision made in Python build flow) to only allow passing CMake overriding settings via a toml file (via -C--local=...). Oftentimes, CMake variables need to be set due to environment conditions such as the toolchain used, installation location of libraries, etc. It is a bit cumbersome to always have to create toml files for one-time passing of such settings.

It gets particularly tricky if there is a combination of different kinds of settings that we may already have separate tomls for: e.g. combining settings for finding libraries/compilers with settings such as the build type (Debug/Release/...).

tttapa commented 10 months ago

No particular reason, other than the limited development time available :) up to now, other features have simply taken precedence.

I agree that it should eventually be supported, but I intend to refactor the current configuration file parser first. Currently, it lacks a good strategy to combine options from different sources: e.g. when combining a CMake -D option that is actually a list (e.g. CMAKE_PREFIX_PATH), right now it only keeps the value from the source that was added last, whereas it would make more sense to merge the different lists.

For example, Conan allows you to explicitly specify how to merge options, but this is not something that TOML supports. Adding support for this would require switching to a different (or alternative) file format for local files, and would require writing a command-line parser that can deal with the configuration tree data structure.

In the short term, I think adding a -D flag (to set CMake variables from the command line) would definitely be a possibility, albeit rather limited.


To make the lack of command-line options slightly more bearable, I usually use a Bash heredoc. For example:

cat > ninja-debug-release.cross.toml <<- EOF
[cmake]
config = ["Debug", "Release"]
generator = "Ninja Multi-Config"
[cmake.options]
CMAKE_FIND_ROOT_PATH = "$staging/pybind11;$staging/eigen-master;$staging/casadi"
USE_GLOBAL_PYBIND11 = "On"
ALPAQA_WITH_CUTEST = "On"
EOF
python -m build -w . \
    -C--cross="$PWD/ninja-debug-release.cross.toml" \
    -C--cross="$staging/$host.py-build-cmake.cross.toml"

Some options can also be passed to CMake using environment variables (https://cmake.org/cmake/help/latest/manual/cmake-env-variables.7.html).

Another possibility is to use CMake presets (https://cmake.org/cmake/help/latest/manual/cmake-presets.7.html), where you define the options you need, and then simply pass that preset to py-build-cmake. It does require configuration files, though.

SanderVocke commented 10 months ago

Thanks for the explanation. A heredoc is a good solution for now! If it's OK with you, I'll leave the issue open for future reference.