Add option in cpt.py to control the number of cores used during make

root-project / cling

The cling C++ interpreter

Other

3.52k stars 269 forks source link

Add option in cpt.py to control the number of cores used during make #391

Open sudo-panda opened 3 years ago

sudo-panda commented 3 years ago

Running make with all cores while building might not be feasible for many. Like in my case the linker crashes due to the lack of memory when all my cores are being used by make. One can obviously edit the file to force the number of cores but having an argument to control the same would be better imo.

reikdas commented 3 years ago

Have you tried -DLLVM_PARALLEL_LINK_JOBS=<some value> via the cmake extra args flag?

sudo-panda commented 3 years ago

@reikdas thanks for the reply, I didn't know about that flag. I tried using that but the linker still crashes. :(

Also my computer freezes up frequently during the installation and restricting make to use a particular number of cores seems to be the way to fix that.

SimeonEhrig commented 3 years ago

I'm not sure, which options the cpt.py tool uses, because I build cling via cmake. But my experience are:

if you build cling with static libraries (default of cmake) it consumes much memory
debug builds increases the memory consumption of the linker very much
simultaneous multi threading can causes easily a out-of-memory problems (after a long freeze on Linux)

For example, our workstation system has a 14 core Intel Xeon with SMT (28 threads) and 128 GB ram. I use 28 threads for compiling and 14 threads for linking. In the worst case (static libraries and debug build) all linkers consumes together 70GB RAM on the same time.

sudo-panda commented 3 years ago

@SimeonEhrig it's the same reason that I think supporting an option to control the number of cores used to build while using cpt.py is a good idea.