search

garu / App-cpanminus-reporter

stand-alone CPAN Testers client (for cpanminus and friends)
13 stars 13 forks source link

unable to differentiate actual tests and fake tests being printed inside tests #40

Open garu opened 7 months ago

garu commented 7 months ago

(originally submitted by @karenetheridge on RT#100337)

CPAN.pm has tests that output "Result: PASS", which are parsed by cpanm-reporter as PASS even if the test itself fails.

$ cpanm --mirror-only --mirror http://mirrors.gossamer-threads.com/CPAN/ CPAN; cpanm-reporter
CPAN
--> Working on CPAN
Fetching http://mirrors.gossamer-threads.com/CPAN/authors/id/A/AN/ANDK/CPAN-2.05.tar.gz ... OK
Configuring CPAN-2.05 ... OK
Building and testing CPAN-2.05 ... FAIL
! Installing CPAN failed. See /Users/ether/.cpanm/work/1415990050.50326/build.log for details. Retry with --force to force install it.
sending: (http://mirrors.gossamer-threads.com/CPAN/authors/id/A/AN/ANDK/CPAN-2.05.tar.gz, ANDK, CPAN-2.05, PASS)
garu commented 7 months ago

There really seems to be no way around this until we are able to get some information from cpanm on what distribution is being tested and what was the output.

garu commented 7 months ago

there is still hope if we manage to tap the actual tests being run. This would allow us to monitor and report independently of cpan clients!

That is the dream (/cc @exodist @leont)

michal-josef-spacek commented 6 months ago

I touch this issue today. The installation directory hasn't rights to install files. cpanm-reporter:

woman:~> cpanm Mo::utils --reinstall
--> Working on Mo::utils
Fetching http://www.cpan.org/authors/id/S/SK/SKIM/Mo-utils-0.27.tar.gz ... OK
Configuring Mo-utils-0.27 ... OK
Building and testing Mo-utils-0.27 ... FAIL
! Installing Mo::utils failed. See /home/skim/.cpanm/work/1716469591.2336625/build.log for details. Retry with --force to force install it.
woman:~> cpanm-reporter
sending: (http://www.cpan.org/authors/id/S/SK/SKIM/Mo-utils-0.27.tar.gz, SKIM, Mo-utils-0.27, PASS)

cat /home/skim/.cpanm/work/1716469591.2336625/build.log:

[SKIP]
t/Mo-utils/12-check_number.t ........... ok
t/Mo-utils/13-check_mumber_min.t ....... ok
t/Mo-utils/14-check_number_of_items.t .. ok
t/Mo-utils/15-check_number_range.t ..... ok
t/Mo-utils/16-check_regexp.t ........... ok
t/Mo-utils/17-check_required.t ......... ok
t/Mo-utils/18-check_string.t ........... ok
t/Mo-utils/19-check_string_begin.t ..... ok
t/Mo-utils/20-check_strings.t .......... ok
All tests successful.
Files=20, Tests=130,  2 wallclock secs ( 0.06 usr  0.03 sys +  1.38 cusr  0.24 csys =  1.71 CPU)
Result: PASS
Manifying 1 pod document
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
ERROR: Can't create '/home/skim/perl5/lib/perl5/Mo'
Do not have write permissions on '/home/skim/perl5/lib/perl5/Mo'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
 at -e line 1.
make: *** [Makefile:644: pure_site_install] Chyba 13
-> FAIL Installing Mo::utils failed. See /home/skim/.cpanm/work/1716469591.2336625/build.log for details. Retry with --force to force install it.