search

jrmarino / synth

Next D/Ports build tool for live systems (Alternative for Portmaster and Portupgrade tools)
ISC License
255 stars 15 forks source link

feature request: Redistribute the idle CPU cores (jobs per builder) among the remaining builders #124

Open Ximalas opened 6 years ago

Ximalas commented 6 years ago

When builders are placed in shutdown, try to redistribute the idle CPU cores (jobs per builder) among the remaining builders to speed up building of the remaining packages.

vasi commented 7 months ago

It ought to be possible using the jobserver feature of make, which is the same thing that allows recursive make to work. See the great description of how GNU make does this.

Unfortunately, bsdmake just has a -J option which is documented as "private". The actual format isn't hard to discover. Plausibly if we set MAKEFLAGS to include "-J X,Y" for bsdmake, and "--jobserver-auth X,Y" for GNU make, they'd inherit that and we could share a max number of jobs across all builders.

jrmarino commented 7 months ago

no, that's mixing topics. Synth allocates cpu jobs to each build. For example. lets say it has 4 builders with 3 jobs each. Each builder is limited to 3 jobs, no matter if it has 64 cpus available.

The original poster is saying, "hey, you allocate 12 totals jobs (amongst 4 builders), so if only 2 builders are active, allocate 6 jobs to each of them".

The second suggestion is based on GNU make which is a fraction of the builds. Many ports don't use make at all. Many more use BSD make over GNU make.

Unless told not to, Synth passed -j to both make and gnu make to limit the jobs it spawns.

I guess technically the original request wouldn't be impossible to do. It's just that I'm not actively developing synth anymore. I did some rare work yesterday to support subpackages so it's basically in maintenance mode.