Perl-Toolchain-Gang / CPAN-Meta

Specifications for CPAN distribution META files
36 stars 40 forks source link

specific dependency phases for "extra" test grades. #108

Open kentfredric opened 8 years ago

kentfredric commented 8 years ago

I thought I'd filed a bug on this somewhere / talked about it enough for people to see it, but I can't figure where I wrote it last and it might have never happened. So.

The short of it is, I feel a growing need to be able to, in metadata, declare dependencies slightly more effectively than a giant nebulous "development requires".

Then, when these stages are formalised and properly integrated into various tools, we can toolchain up equivalent of this mess, but without requiring lots of dzil glue


if ($ENV{AUTOMATED_TESTING} or $ENV{AUTHOR_TESTING} ) {
     $TEST_REQUIRES = { %$TEST_REQUIRES, %$AUTHOR_TESTING_REQUIRES }
    $tests .= " xt/author/*"
}
if ($ENV{AUTOMATED_TESTING} or $ENV{RELEASE_TESTING} ) {
     $TEST_REQUIRES = { %$TEST_REQUIRES, %$RELEASE_TESTING_REQUIRES }
    $tests .= " xt/release/*"
}
if ($ENV{AUTOMATED_TESTING} or $ENV{EXTENDED_TESTING} ) {
     $TEST_REQUIRES = { %$TEST_REQUIRES, %$EXTENDED_TESTING_REQUIRES }
    $tests .= " xt/smoke/*"
}

Because "Hey, lets make automated testing install dzil just to run authortests on automated testing boxes".

Advantages to running release/author tests on AT Boxes:

Leont commented 8 years ago

The details of getting this right aren't trivial at all. Too bad you're not going to be un Rugby :-/

haarg commented 8 years ago

This is definitely something I'd like to have in some form, even if just as a custom field. Currently many Dist::Zilla plugins add themselves to develop requires. In some sense this is correct, but it also doesn't pertain to anything that's actually part of the dist. Having a way to pull in the prereqs needed for xt tests that ship with the dist without including the prereqs needed to generate the dist is very useful.

I'm currently using a 'x_test' prereq phase for this. I haven't found the granularity of separate xt subdirectories particularly useful.

kentfredric commented 8 years ago

I haven't found the granularity of separate xt subdirectories particularly useful.

To an extent I agree, there will surely be some userbase ( and indeed, testing targets ) which may want a "Run them all" phase, and thus, a merged view of all those dependencies is logical for that.

But there's likely going to be user targets other than that, for instance:

I can totally understand why a CPAN smoker might want to run only xt/smoke/ in addition to t/, but not run xt/author or xt/release

And for contributor workflows, "Smoke" might pull in things like reverse-dependencies for regression testing, and so you don't want to be telling them "Hey, just to run author tests, you need to install these parts of CPAN you're not going to care about"

The point is breaking down the effort required into teirs, so people can invest as much as they want ( that is, all tests, all dependencies ) or as little as they want ( just the code, just the code and the author tests, ... etc )

xdg commented 8 years ago

I'm currently using a 'x_test' prereq phase for this

I'd like to see better arbitrary phase support in "installdeps" targets. E.g.

$ cpanm --installdeps --with_phase=x_test .

If that existed, then I think it's a matter of someone writing tooling to put deps at whatever granularity into META.

kentfredric commented 8 years ago

In practice that also means you want custom phase support in CPAN::Meta::Prereqs, and have to tie the data from CPAN::Meta into it. Because having those phases is not so useful if you can't add data to it.... and all the tools that presently exist for "Communicate dependences through authorship tools" rely on those parts to do that.

=>

x_test => Seems useful if you're doing hand edited small dists, but not practical for something like DZil.

kentfredric commented 8 years ago

Oh, and I suspect there'll need to be some effort somewhere to support CPAN::Meta merging those deps into the test prereqs intelligently. Because you're probably going to want to do that in EUMM one day.