CPAN-Reporter uninstallable on FreeBSD and other OSes due to test failure in dependency

[jkeenan]

CPAN-Reporter is not currently installable (at least not without resorting to --force at certain points) on FreeBSD and other operating systems. Because of that, it is not currently possible to set up new CPANtesters instances, whether for casual use -- send a report each time you use cpan -- or for more serious use like the comprehensive rigs that Andreas Koenig, Slaven Rezic and Chris Williams have.

In addition, Blead-Breaks-CPAN-detection programs, such as the one I've been running for several months on Linux (see, e.g., https://www.nntp.perl.org/group/perl.perl5.porters/2018/03/msg250058.html), will not reach CPAN-Reporter on affected operating systems. That means that distributions dependent on CPAN-Reporter will not be reached either, and potential BBC problems in those modules will go undetected in the final stages of preparation of perl-5.28.0 release.

The problem does not lie in CPAN-Reporter. It lies in File-Copy-Recursive (hereinafter, FCR), upon which CPAN-Reporter has a dependency -- as do hundreds of other CPAN modules, including the entirety of DateTime and Dist-Zilla. By one calculation, FCR is in the 129th farthest upstream position on the CPAN river. None of those libraries are currently installable on FreeBSD (unless you resort to a force-install of File-Copy-Recursive).

Relevant links:

sco: http://search.cpan.org/~dmuey/File-Copy-Recursive-0.40/ met: https://metacpan.org/release/File-Copy-Recursive bug: https://rt.cpan.org/Ticket/Display.html?id=123964 fmx: http://fast-matrix.cpantesters.org/?dist=File-Copy-Recursive pr: https://github.com/drmuey/p5-File-Copy-Recursive/pull/11/files

As one can see from the bug report cited above, the problem -- a single failure in FCR's test suite reported by Slaven (of course!) in December. FCR's author noted the report early in January. Tom Hukins provided a patch (via p.r.) later in January. In the course of examining the result of my Test-Against-Dev run of perl-5.27.10 in March, I confirmed the bug and, at the beginning of this month, confirmed that Tom's patch would enable FCR to pass all its tests and be installable on FreeBSD. That would in turn (a) permit CPAN-Reporter to be installed on FreeBSD and re-open an important source of CPANtesters reports, and (b) enable all of FCR's revdeps to be reachable via Test-Against-Dev and similar programs when run on OSes other than Linux.

If FCR's maintainer does not take action before the release of perl-5.28.0 in less than five weeks from now, then those Perl users on non-Linux OSes looking to upgrade to 5.28 and install modules via 'cpan' will be out of luck. To prevent that from happening, CPAN-Reporter's test suite -- specifically t/lib/Helper.pm -- should be patched to include stripped-down substitutes for the two FCR subroutines, fcopy() and dircopy(), used in CPAN-Reporter's test suite. I will be submitting a p.r. which does precisely that. The fcopy() and dircopy() functions in that p.r. are not suitable for general use, but they are suitable for a use like that in CPAN-reporter's test suite, where we are only concerned with directories and files, know in advance all the directories and files we want to copy and don't need all the bells and whistles which FCR's versions offer.

Please contact me, especially before and during the Perl Toolchain Summit, to discuss this problem.

Thank you very much. Jim Keenan

[jkeenan]

Today I uploaded to CPAN a new module, File::Copy::Recursive::Reduced, which provides better versions of fcopy() and dircopy() than those I wrote into CPAN-Reporter's test suite in https://github.com/cpan-testers/CPAN-Reporter/pull/89 yesterday. The dircopy() function, in particular, more closely does what File::Copy::Recursive::dircopy() does. So, if you are going to apply one of these two p.r.s, I would recommend this one.

[karenetheridge]

It would probably be best if File::Copy::Recursive got a new release with the proper fix, but it seems less and less likely that that will happen in time :(

[jkeenan]

The particular problem with File::Copy::Recursive that I addressed with File::Copy::Recursive::Reduced was fixed in yesterday's release of FCR version 0.41.

That version, however, is failing tests on quite a few CPANtesters rigs for other reasons. On the machines where those failures are present, CPAN-Reporter will still be unreachable via cpan or cpanm or during Blead-Breaks-CPAN-focused testing. You may still wish to consider p.r. https://github.com/cpan-testers/CPAN-Reporter/pull/90.

Thank you very much. Jim Keenan

[garu]

@jkeenan @karenetheridge could you please make sure master works again so we can make a new release?

[jkeenan]

On 04/24/2018 04:43 PM, Breno G. de Oliveira wrote:

@jkeenan https://github.com/jkeenan @karenetheridge https://github.com/karenetheridge could you please make sure |master| works again so we can make a new release?

commit 95eaca7a18a3cb92122217ae43b88c52b3c7bc54 Merge: 6c1c243 268baa0 Author: Breno G. de Oliveira garu@cpan.org Date: Tue Apr 24 22:45:15 2018 +0200

PASS on both Linux and FreeBSD-11.1

[nigelhorne]

FWIW, the version in CPAN fails on DragonFly with Perl 5.28.0:

===(     133;1  11/33  1/? )============================================mv: rename /tmp/7c1YirhPXn/pathrm/ to /tmp/7c1YirhPXn/moved/pathrm/: Not a directory

#   Failed test 'pathrm() detected symlink toggle: bails when below level changes'
#   at t/02.legacy-symtogsafe.t line 107.
#                   'could not toggle dir/symlink (mv): 256
# '
#     doesn't match '(?^:directory .* changed: expected dev=.* ino=.*, actual dev=.* ino=.*, aborting)'
t/04.readonly-dir.t ....... ok
t/05.legacy-pathmk_unc.t .. skipped: Test irrelevant on non-windows OSs
t/02.legacy-symtogsafe.t .. 16/33 mv: rename /tmp/7c1YirhPXn/pathempty/ to /tmp/7c1YirhPXn/moved/pathempty/: Not a directory

.....

Running make for D/DA/DAGOLDEN/CPAN-Reporter-1.2018.tar.gz
Warning: Prerequisite 'File::Copy::Recursive => 0.35' for 'DAGOLDEN/CPAN-Reporter-1.2018.tar.gz' failed when processing 'DMUEY/File-Copy-Recursive-0.44.tar.gz' with 'make_test => NO'. Continuing, but chances to succeed are limited.

[karenetheridge]

Warning: Prerequisite 'File::Copy::Recursive => 0.35' for 'DAGOLDEN/CPAN-Reporter-1.2018.tar.gz' failed when processing 'DMUEY/File-Copy-Recursive-0.44.tar.gz' with 'make_test => NO'. Continuing, but chances to succeed are limited.

I think this ^^ line reveals why I receive so many FAIL reports from Nigel Horne where prerequisites are not satisfied. Nigel, please stop doing that! It is polluting the statistics and the resulting reports are quite useless: all it indicates is that the prerequisite is a legitimate one and the lack of the module does indeed result in a failure.

[karenetheridge]

Then fix the code so it doesn't attempt to do that. I run everything straight out of the box with no changes!

CPAN clients are not necessarily calibrated to the usecase of someone who sends thousands of automated reports a day.

Also, please try to be more friendly, a better response would have been to say "something is up, if you change this is may help".

After multiple years of appealing to you to reduce the number of erroneous reports, my patience has run a bit thin.

[jkeenan]

On 04/24/2018 04:43 PM, Breno G. de Oliveira wrote: @jkeenan https://github.com/jkeenan @karenetheridge https://github.com/karenetheridge could you please make sure |master| works again so we can make a new release? commit 95eaca7 Merge: 6c1c243 268baa0 Author: Breno G. de Oliveira garu@cpan.org Date: Tue Apr 24 22:45:15 2018 +0200 PASS on both Linux and FreeBSD-11.1

The problem for which I originally filed this ticket more than 3 years ago has been resolved for the OSes I myself was concerned about.

Prerequisite File-Copy-Recursive, however, is still failing on DragonflyBSD. My substitute library, File-Copy-Recursive-Reduced, may be of benefit here -- but I myself don't have access to that OS to verify that.

It's been quite a few years since a new version of CPAN-Reporter was released. Please consider addressing outstanding tickets and issuing a new release.

Thank you very much. Jim Keenan

[jkeenan]

The problem for which I originally filed this ticket more than 3 years ago has been resolved for the OSes I myself was concerned about.

Prerequisite File-Copy-Recursive, however, is still failing on DragonflyBSD. My substitute library, File-Copy-Recursive-Reduced, may be of benefit here -- but I myself don't have access to that OS to verify that.

The problem with File::Copy::Recursive on Dragonfly BSD should not be an obstacle to either closing this ticket or making a new release. @haarg provided a workaround in this commit:

commit 5ed42c23537443d8b9fe053bcc747add723917d1
Author:     Graham Knop <haarg@haarg.org>
AuthorDate: Tue Apr 17 15:48:56 2018 +0200
Commit:     Graham Knop <haarg@haarg.org>
CommitDate: Tue Apr 17 16:13:33 2018 +0200

    avoid File::Copy::Recursive

    For copying files, we can just use File::Copy.  For copying directories,
    we can reimplement what we need using File::Find and File::Copy.

Thank you very much. Jim Keenan

[jkeenan]

The problem for which I originally filed this ticket more than 3 years ago has been resolved for the OSes I myself was concerned about. Prerequisite File-Copy-Recursive, however, is still failing on DragonflyBSD. My substitute library, File-Copy-Recursive-Reduced, may be of benefit here -- but I myself don't have access to that OS to verify that.

The problem with File::Copy::Recursive on Dragonfly BSD should not be an obstacle to either closing this ticket or making a new release. @haarg provided a workaround in this commit:

commit 5ed42c23537443d8b9fe053bcc747add723917d1
Author:     Graham Knop <haarg@haarg.org>
AuthorDate: Tue Apr 17 15:48:56 2018 +0200
Commit:     Graham Knop <haarg@haarg.org>
CommitDate: Tue Apr 17 16:13:33 2018 +0200

    avoid File::Copy::Recursive

    For copying files, we can just use File::Copy.  For copying directories,
    we can reimplement what we need using File::Find and File::Copy.

We'll need a new CPAN release of CPAN-Reporter for @haarg's p.r. to have beneficial effect, but this ticket is closable.