davidchisnall
commented
2 years ago It's very difficult to get right. I had some thoughts about whether we could use ptrace for this and attach to each process to catch new thread / process creation, with a fixed-sized pool of allowed tasks, so every jail is told to use N cores, but process creation in the second jail will block if the first jail has used all N cores, and so on.
This is made a lot more difficult by lld, which uses one thread per core. The LLVM ports will spawn a single lld instance at a time but if you're building three versions of LLVM at the same time on an 8-core machine then you can have 8 different instances of LLD running, each with 8 threads and each consuming a lot of memory.
This could possibly be addressed by trapping on execve and reimplementing something like Ninja's job pools to say that lld counts as N compile jobs.
bdrewery
At this moment poudriere is compiling llvm11,llvm12,llvm13 simultanuously. There are 3 "ninja" processes running. I have an 8 core cpu. There are 400 ports waiting in the queue on the finishment of the compilation of the llvms. My poudriere.conf contains ALLOW_MAKE_JOBS=YES My make.conf contains MAKE_JOBS_NUMBER=8 Yet 5 cores are unused which is a waste of time and energy. An intelligent system would spawn 3 processes for the build of llvm11 , 3 processes for the build of llvm12 , and 2 processes for the build of llvm13. So that the parallel build uses all cores optimal. I don't know if this is technical even possible or complex or not, but it would be a nice feature.