Debug build setting not passed to CMake build config in Conda-forge

Zeitsperre commented 8 months ago

Hi,

I've been searching through the documentation at conda-forge, scikit-build, scikit-build-core, CMake, StackOverflow, and I feel like I'm very close to finding an answer, but I'd like to know for certain whether this is an issue with the build engine, my configuration, the conda-forge configuration or something else.

Background

I have a Python package (raven-hydro) that is a relatively simple wrapper for a binary written in C++ (RavenHydroFramework). The Python package has been using scikit-build-core for a while now, and it has been working very well. I have been able to cross-compile things very easily. Thanks again for this great tool!

Now, the latest version of RavenHydroFramework must be compiled in with -DCMAKE_BUILD_TYPE=Debug. Getting this to work for PyPI-based packages has been very simple:

[tool.scikit-build.cmake]
build-type = "Debug"

Problem

This package is also on conda-forge, and in previous versions, there were zero issues building for macOS/Windows/Linux. According to the conda-forge documentation, several CMAKE_ARGS are set on environment activation. This shouldn't be an issue, as the arguments passed to CMake from Python should override these settings. Unfortunately, that hasn't been my experience.

Despite the fact that the build logs on the conda-forge Azure pipelines showing messages that give me the impression that the CMAKE_BUILD_TYPE from the environment is being ignored, e.g.

raven-hydro_1709925961683/_build_env/bin/x86_64-conda-linux-gnu-strip', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER', '-DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=ONLY', '-DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=ONLY', '-DCMAKE_FIND_ROOT_PATH=/home/conda/feedstock_root/build_artifacts/raven-hydro_1709925961683/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold;/home/conda/feedstock_root/build_artifacts/raven-hydro_1709925961683/_build_env/x86_64-conda-linux-gnu/sysroot', '-DCMAKE_INSTALL_PREFIX=/home/conda/feedstock_root/build_artifacts/raven-hydro_1709925961683/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold', '-DCMAKE_INSTALL_LIBDIR=lib', '-DCMAKE_PROGRAM_PATH=/home/conda/feedstock_root/build_artifacts/raven-hydro_1709925961683/_build_env/bin;/home/conda/feedstock_root/build_artifacts/raven-hydro_1709925961683/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/bin']
2024-03-08T19:28:36.3745077Z   2024-03-08 19:28:36,346 - scikit_build_core - WARNING - Unsupported CMAKE_ARGS ignored: -DCMAKE_BUILD_TYPE=Release

My package still doesn't seem to build with the proper CMAKE_BUILD_TYPE flag set. Oddly enough, this issue isn't occurring across all platforms; On Windows, it would appear the mechanisms are working as intended, and I believe this may have something to do with the fact that scikit-build-core passes configuration commands to CMake/MSVC via a different process than used for macOS and Linux (if I understand correctly).

I've traced the issue to one line that gives me the impression that the configurations are not being translated 100% to the underlying CMake call.

Conda-forge Windows build log:

2024-03-08T19:36:49.0016833Z   *** Building project with Visual Studio 17 2022...
2024-03-08T19:36:49.0017720Z   2024-03-08 19:36:48,986 - scikit_build_core - DEBUG - RUNENV - changes since last run only:
2024-03-08T19:36:49.0018569Z 
2024-03-08T19:36:49.0019587Z   2024-03-08 19:36:48,986 - scikit_build_core - INFO - RUN: D:\bld\raven-hydro_1709926259969\_build_env\Library\bin\cmake.EXE --build build -v --config Debug

Conda-forge macOS build log:

2024-03-08T19:38:16.5143560Z   *** Building project with Unix Makefiles...
2024-03-08T19:38:16.5146150Z   2024-03-08 19:38:16,511 - scikit_build_core - DEBUG - RUNENV - changes since last run only:
2024-03-08T19:38:16.5147020Z 
2024-03-08T19:38:16.5148350Z   2024-03-08 19:38:16,511 - scikit_build_core - INFO - RUN: /Users/runner/miniforge3/conda-bld/raven-hydro_1709926231787/_build_env/bin/cmake --build build -v

Conda-forge Linux build log:

2024-03-08T19:28:37.0290022Z   *** Building project with Unix Makefiles...
2024-03-08T19:28:37.0295585Z   2024-03-08 19:28:37,027 - scikit_build_core - DEBUG - RUNENV - changes since last run only:
2024-03-08T19:28:37.0300370Z 
2024-03-08T19:28:37.0306218Z   2024-03-08 19:28:37,027 - scikit_build_core - INFO - RUN: /home/conda/feedstock_root/build_artifacts/raven-hydro_1709925961683/_build_env/bin/cmake --build build -v

Questions

Am I correct in believing that arguments set within the [tool.scikit-build.cmake.define] block can override the environment variables set by conda-forge?
- For context, in addition to setting [tool.scikit-build.cmake] build-type = "Debug", I've also set the following to be certain that my build configuration is being seen by CMake:
pyproject.toml:
```
[tool.scikit-build.cmake.define]
CMAKE_BUILD_TYPE = {env="RAVEN_HYDRO_CMAKE_BUILD_TYPE", Default="Debug"}
```
CMakeLists.txt:
```
# Set default build type in CMake if skbuild is set
message(STATUS "CMAKE_BUILD_TYPE set to '${CMAKE_BUILD_TYPE}'")
```
Is there a functional difference between setting CMake flags via pyproject.toml, via the CLI, or environment variables when it comes to these flags being passed along to build functions spawned by pip/scikit-build-core? Is one method more persistent/permanent than another? What is the priority order when it comes to these methods?
What would you suggest is the best way to ensure that --config Debug is passed to CMake on conda-forge?

Additional Information

raven-hydro testing branch: https://github.com/Ouranosinc/raven-hydro/pull/36 raven-hydro testpypi dev package: https://test.pypi.org/project/raven-hydro/0.3.1.dev0/ conda-forge pull request (currently using v0.3.1-dev): https://github.com/conda-forge/raven-hydro-feedstock/pull/25

henryiii commented 8 months ago

A quick comment: Yes, the process is different between Windows and Unix. Specifically, it's actually based on the generator - multi-config generators (default on Windows) have the flags you are seeing. You can use a higher verbosity level to see the full file of presets that is passed to CMake.

I'll try to look into this soon, before next release.

henryiii commented 7 months ago

I see -DCMAKE_BUILD_TYPE:STRING=Debug in the configure line for Unix in your logs. It seems to be filtering it from CMAKE_ARGS and setting it in the configure step, as is correct for single config generators. You are getting undefined reference to ExitGracefully, is that related to the build type?

henryiii commented 7 months ago

Where are you linking to the RavenHydroFramework library or including that in your build? I see you adding that to an executable, but not to the Python target. ExitGracefully is defined there. When you build normally, you won't notice that, since Python extension modules usually ignore missing symbols when building on Unix[^1]. But I'd guess conda-build is checking for missing symbols?

[^1]: There's a way to do this, but it's tricky and we've never set it up for pybind11. It's in nanobind though. You basically have to give the compiler the list of symbols without linking to the binary so it can detect any missing symbols not in the list.

Zeitsperre commented 7 months ago

@henryiii Thanks for taking the time to look this over!

I see -DCMAKE_BUILD_TYPE:STRING=Debug in the configure line for Unix in your logs. It seems to be filtering it from CMAKE_ARGS and setting it in the configure step, as is correct for single config generators. You are getting undefined reference to ExitGracefully, is that related to the build type?

Unfortunately, yes. The build type has an impact on the availability of the functions. I'm not an expert in C/C++ but for the time being, the only way to build the executable is in debug mode. The source code for the project can be found here (Relevant issue: https://github.com/CSHS-CWRA/RavenHydroFramework/issues/24). I'm not sure how challenging it would be to ensure that the missing symbols are carried over to the release build.

Where are you linking to the RavenHydroFramework library or including that in your build? I see you adding that to an executable, but not to the Python target. ExitGracefully is defined there. When you build normally, you won't notice that, since Python extension modules usually ignore missing symbols when building on Unix. But I'd guess conda-build is checking for missing symbols?

The library is fetched via git and the source file that performs the linking with Pybind11 can be found here. There are plans to eventually link some functionality at a library level, but the work is very early in development. I wish I knew more about the conda-forge internals, but I'm already out of my depth.

scikit-build / scikit-build-core