floooh / fips

High-level build system for distributed, multi-platform C/C++ projects.
MIT License
471 stars 82 forks source link

Using --parallel in cmake does not build in parallel within projects #273

Closed kaffeewolf closed 2 years ago

kaffeewolf commented 3 years ago

Not really a fips issue, probably more like a CMake issue, but the switch to using cmake build --parallel in 3a55aa3472014b448fabb9f33a07f1b043bf3ae2 results in poor build times when using fips build from the commandline on Windows with MSVC. It appears the build is parallel per project only, so having a large project and a lot of small ones will become very slow as the large project is built on a single core only. In my case build times went from 110s to > 300 (thats when i aborted).

Building in Visual Studio it seems to work fine however.

floooh commented 3 years ago

Hmm yeah, that's sort of a known issue (https://github.com/floooh/fips/issues/270).

You could try "fixing" this line (remove the --parallel {} and num_jobs and see if this changes anything (I doubt it though - but if it does we could probably remove the --parallel when the Visual Studio generator is used):

https://github.com/floooh/fips/blob/master/mod/tools/cmake.py#L83

AFAIK MSVC has two types of parallelism, one between targets (which is probably what the --parallel switch uses, and a separate switch /MP (called "Multi-processor Compilation" in the Visual Studio options.

The /MP flag is activated in the Windows "toolchain file" used by fips, here:

https://github.com/floooh/fips/blob/3df5202aac0dfbd1dafd6e0b53b9c531c33a6742/cmake-toolchains/windows.cmake#L57

...and here:

https://github.com/floooh/fips/blob/3df5202aac0dfbd1dafd6e0b53b9c531c33a6742/cmake-toolchains/windows.cmake#L63

...so I would expect that building within a single target is also parallelized.

mattiasljungstrom commented 2 years ago

I noticed this today as well, in my project building with --parallel takes 34s and without 13s. Building with msbuild directly takes 12s.

mattiasljungstrom commented 2 years ago

Building inside the VS IDE is even faster for some reason, around 9s for this project.

floooh commented 2 years ago

Closing this because of PR #289.