search

gobolinux / Compile

The GoboLinux build tool
52 stars 15 forks source link

Safely make use of multi-threading by default #59

Closed Nuc1eoN closed 2 months ago

Nuc1eoN commented 2 months ago

Prior to this commit Compile would run Make single-threaded by default. This commit ensures that Compile runs Make with all CPU threads available minus 4 (if possible).

Leaving 4 threads available seems to be a good compromise as a default setting. The setting is backwards compatible and will run single-threaded on systems with 5 or less processing units available.

mwh commented 2 months ago

I am pretty leery of a configuration file that runs code every time it's loaded. Maybe this could be something the installer could make a default entry in Compile.conf for (though I think the instructions have said to set this to taste for a while). If it's part of the tool itself then this seems like the wrong place for it to go, rather than inside the actual executable somewhere.

hishamhm commented 2 months ago

Beware that not all packages work with parallel make. If this is merged, expect having to patch some recipes to override it.

I used to have -j enabled in my personal Compile config, but in the knowledge that eventually I'd have to manually disable it and retry failed builds.

Nuc1eoN commented 2 months ago

The motivation behind this was to provide a sensible default.

Many users, when they are fresh to gobo, may not know about this setting and have unnecessary long build times.

But also personally, I happened to have fresh GoboLinux installs where I simply forgot to set my core count for a while (it is not immediately obvious unless you open htop).

In general yes, the wiki does mention to configure the threads count to own liking.

I am pretty leery of a configuration file that runs code every time it's loaded. Maybe this could be something the installer could make a default entry in Compile.conf for (though I think the instructions have said to set this to taste for a while).

Good point actually, hadn't thought about this as an issue. But indeed, this might be a better fit for the Installer.

Beware that not all packages work with parallel make. If this is merged, expect having to patch some recipes to override it.

Frankly, I wasn't aware this is still much of an issue. I have now compiled quite a lot of packages under GoboLinux with multi-threading enabled and never faced such an issue.

Do you know of any notable packages specifically that fail compile multi-threaded nowadays??

In general I believe "in an ideal world" multi-threading should be set as default, and offending packages patched in the recipe.

However since I do not know how many recipes would be affected, probably lets not make this a default as of now.

Thank you for pointing out the potential issues :)

lucasvr commented 2 months ago

Do you know of any notable packages specifically that fail compile multi-threaded nowadays??

There are several. You can get a taste here: https://github.com/search?q=repo%3Agobolinux%2FRecipes+-j1&type=code

Nuc1eoN commented 2 months ago

@lucasvr thanks lucas! And I didn't even know GitHub had such a handy in-text search. That will come in handy in the future:)