search

Perl-Toolchain-Gang / cpan-api-buildpl

Documentation of the API for using Build.PL
11 stars 3 forks source link

--jobs? #4

Open Leont opened 11 years ago

Leont commented 11 years ago

We live in a multi-core world. I'd like Build.PL to enable that in a sane way. Therefor I suggest we add a --jobs argument. This would be suggestion for the build tool to use a certain number of processes/threads when doing its job, similar to make -j.

Adding support for this in the testing phase is fairly easy (in both Module::Build and Module::Build::Tiny), and a future implementation may also implement it for other phases.

dagolden commented 11 years ago

+1, though I might suggest it as a "request" for parallelism and a client may choose to ignore it.

ribasushi commented 11 years ago

I discussed this very topic with @andk in Lancaster, and he very quickly convinced me that this is the wrong approach. That is - having dists label themselves as "I am -j$large ready" is counterproductive. Instead we simply strive to get the toolchain -j$large-clean. And we are much closer than many of you think. For instance the DBIC smoker runs everything under -j32 (and if it fails reruns things without it to save me red). The only intermittent (like all -j-related) issue that I know of currently is Uri's File::Slurp (https://rt.cpan.org/Ticket/Display.html?id=88968). The rest: https://travis-ci.org/dbsrgits/dbix-class/jobs/12412649#L74

So I think having any sort of communication mechanism is the wrong way to go, instead we need more smokers with irrevocable -j set (I don't think we have many at all... looks at @bingos)

haarg commented 11 years ago

I'm not sure how any of that conflicts with this. The proposed --jobs argument would be a way to tell Build.PL-using dists to use multiple jobs for building (not testing) and could then be set in PERL_MB_OPT.

ribasushi commented 11 years ago

I am a muppet - misunderstood the intent of the proposal... Hmmm (offtopic) does this mean that when I say HARNESS_OPTIONS=jX the EUMM build phase is not running in parallel?

Leont commented 11 years ago

+1, though I might suggest it as a "request" for parallelism and a client may choose to ignore it.

Yeah, that was the way I meant it.

Hmmm (offtopic) does this mean that when I say HARNESS_OPTIONS=jX the EUMM build phase is not running in parallel?

That is correct. HARNESS_OPTIONS only affects Test::Harness/TAP::Harness::Env, it doesn't affect make/Build.