build-cpp / cmkr

Modern build system based on CMake and TOML.
https://cmkr.build
MIT License
431 stars 27 forks source link

Implement CMake integration + bootstrapping #2

Closed mrexodia closed 3 years ago

mrexodia commented 3 years ago

This is an initial version that implements some kind of self-hosting of cmkr from within cmake. The idea is that doing:

mkdir build && cd build
cmake ..

Will automatically build cmkr and regenerate CMakeLists.txt whenever you run cmake or when you update cmake.toml.

I tested on Windows 10 with Visual Studio 2019 and WSL2 with Ubuntu 20.04 and it works super well!

The bootstrapper is optional and will only be used if you have cmkr.cmake next to CMakeLists.txt. Likely cmkr gen has to be updated to automatically generate this file (with an option), but that can be done easily.

Design considerations

I explicitly designed this integration in such a way that it will not search the user's PATH, but instead always rebuild the top-level cmkr from scratch (all sub projects will use the version built by the root project). This is done because you can then pin to a specific cmkr version if you want and otherwise it will always use the latest version. You could change this, but then you have the issue that a package manager's cmkr version can basically be anything and users will run into compatibility issues later.

To override the cmkr.exe (only really useful for development purposes) use -DCMKR_EXECUTABLE=/path/to/cmkr.

The main downside is that whatever compiler cmake selects without specifying it currently has to support std::filesystem (most linux distributions do not have this (you have to do CC=gcc-10 CXX=g++-10 cmake ..).

The fix would be to have cmkr depend on the lowest c++ standard and cmake version as humanly possible, to allow a seamless bootstrapping process. Another idea is to rewrite cmkr in cmake itself, but I understand why you wouldn't want to do this. I did a quick check and it should be quite easy to switch to https://github.com/gulrak/filesystem to go down to c++11.

Additionally cross compiling is probably not working at all, but I don't use it so I didn't care. LLVM supports this for llvm-tblgen and you can look at that if you really want cross-compilation support (you have to somehow make CMake select a valid host compiler for that). Likely if somebody does -DCMAKE_C_COMPILER=xxx -DCMAKE_CXX_COMPILER=yyy (eg not use the CC and CXX environment variables) it will work but it's not a supported scenario right now.

Another downside is that because you cannot include() a file that you are currently already in (cmake bug?) I temporarily generate CMakerLists.txt next to CMakeLists.txt, which is then included and some hacks are done to hide this from the user. I'm not entirely sure if this is safe, but because cmkr is generating the CMakeLists.txt in question it should be fine.

Previously I was using configure_file to generate a CMakeLists.txt in the CMAKE_CURRENT_BINARY_DIR, but this could interfere with subprojects that also use cmkr so I decided to generate a temporary file in the source directory instead.

A fix would be to instead generate CmkrGenerated.cmake, move all the bootstrapping code to CMakeLists.txt, which then does include(CMkrGenerated.cmake). That way you can overwrite that file before cmake 'sees' it and make it work without the terrible hacks. You could then even add CMkrGenerate.cmake to the .gitignore and have just a static CMakeLists.txt + cmake.toml checked out.

MoAlyousef commented 3 years ago

Hi Wow. Thanks for the pull request. I can try to use one of the portable filesystem libraries. I can just use a union instead of std::variant since it's an internal implementation.

mrexodia commented 3 years ago

Glad to help and thanks for trying to make the CMake ecosystem more usable!