search

diegoferigo / cmake-build-extension

Setuptools extension to build and package CMake projects
https://pypi.org/project/cmake-build-extension/
MIT License
67 stars 16 forks source link

Add support for parallel build jobs on CMake #37

Closed kdewald closed 1 year ago

kdewald commented 2 years ago

This should help speed up the builds of projects that have a large number of files that need to be compiled.

kdewald commented 2 years ago

This needs https://github.com/diegoferigo/cmake-build-extension/pull/38 landed before we can proceed.

kdewald commented 2 years ago

I'm not really sure about this PR @kdewald, the usage of Ninja was meant as a multiplatform solution that compiles jobs by default in parallel and automatically computes the right number of jobs depending on the hardware.

This PR enforces the default -j1, therefore all the automatic capability of Ninja is hindered.

I'm not very familiar with Windows builds, but on Linux the default generator is -G "Unix Makefiles" that can be selected as soon as #35 lands. In this case, specifying the number of jobs is a nice (optional) feature since by default it uses -j1.

I agree that making the feature optional is better. One weird thing I noticed at work is that for some reason building using Ninja seems to take much longer, so I'm not sure what could be happening there.

diegoferigo commented 2 years ago

One weird thing I noticed at work is that for some reason building using Ninja seems to take much longer, so I'm not sure what could be happening there.

Is your work setup based on Windows? Does your proejct use the gcc or the MSVC compiler? In my limited experience, setting up Ninja with MSVC is not straightforward. If I recall well, there were two alternatives:

  1. Prepare the terminal calling vcvars and then configure the project with -GNinja.
  2. Configure the project with -GNinja in a visual studio terminal (I remember there's a different terminal app in the windows menu just for VS, can't recall the precise name. It already opens with configured env vars).

Please note that the Windows jobs in CI select gcc, not the MSVC compiler:

[...]
  -- The C compiler identification is GNU 11.2.0
  -- The CXX compiler identification is GNU 11.2.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: C:/ProgramData/chocolatey/bin/gcc.exe - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: C:/ProgramData/chocolatey/bin/c++.exe - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Found Python3: C:/hostedtoolcache/windows/Python/3.8.10/x64/python3.exe (found version "3.8.10") found components: Interpreter Development.Module NumPy
  -- Found SWIG: C:/ProgramData/chocolatey/bin/swig.exe (found suitable version "4.0.2", minimum required is "4.0")
  -- Configuring done
  -- Generating done
[...]
diegoferigo commented 2 years ago

I just discovered that Ninja seems the default generator in VS2022:

Because Ninja is designed for fast build speeds instead of flexibility and function, it's set as the default. However, some CMake projects may be unable to correctly build using Ninja. If that occurs, you can instruct CMake to generate a Visual Studio project instead.

diegoferigo commented 1 year ago

Closing this stale PR. Feel free to comment or reopen in case of updates.