search

bloomberg / bde

Basic Development Environment - a set of foundational C++ libraries used at Bloomberg.
Apache License 2.0
1.68k stars 318 forks source link

multithreaded builds causing calamities #263

Closed seanbaxter closed 4 years ago

seanbaxter commented 4 years ago

I usually build tests with -j3, and about 50% of the time that's fine. 50% of the time one of the processes runs out of memory (I assume). This is almost always at stl_vector_test.t.cpp. It also gets unordered map and hash at about the same time, which collectively is extreme load on my system. Can cmake_build.py somehow detect out of memory error so that it can re-issue a failed build process on just a single thread and then resume compilation on multithreads? It's a bummer to have to re-issue the build command once or even twice to resume compilation.

cppguru commented 4 years ago

I have split those test drivers up just recently. Could you test again now?

cppguru commented 4 years ago

Also, just curious. Aren't those multi-process builds?

seanbaxter commented 4 years ago

Splitting the larger tests up helps a lot. Able to push to -j6 in most cases. Ya the're multi-process. Would be hard to imagine compiling the tests on a single process. :)