[win32] File::ShareDir::ProjectDistDir is not fork-aware

[basiliscos]

Here is a small test, which shows the problem under windows:

https://gist.github.com/basiliscos/7792020

c:\basiliscos\perl-watcher-master>prove -l -v t\1.t
t\1.t ..
This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
Dubious, test returned 3 (wstat 768, 0x300)
No subtests run

Test Summary Report
-------------------
t\1.t (Wstat: 768 Tests: 0 Failed: 0)
  Non-zero exit status: 3
  Parse errors: No plan found in TAP output
Files=1, Tests=0,  2 wallclock secs ( 0.14 usr +  0.05 sys =  0.19 CPU)
Result: FAIL

Commenting use File::ShareDir::ProjectDistDir passes test.

Perl version:

c:\basiliscos\perl-watcher-master>perl -v

This is perl 5, version 18, subversion 1 (v5.18.1) built for MSWin32-x86-multi-thread-64int

Copyright 1987-2013, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

[kentfredric]

Strange, there's no code in F:SD:PDD that is inherently fork oriented.

Also, in that test, F:SD:PDD loads long before the fork, and calling "use" inside eval does nothing.

BEGIN {
    print "Begin 1\n";
}

our $cond;

if ( $cond ) {
    print "Cond 1\n";
    eval {
        BEGIN { 
            print "Begin 2\n";
        }
    }
}

Here, the BEGIN phase runs despite the eval never being called, and the output is:

Begin 1
Begin 2

And the same is the case with use, because use is BEGIN { require Module; Module->import( LIST ); }

BEGIN {
    print "Begin 1\n";
}

BEGIN { 
    package A;
    sub import {
        print "Importing A\n";
    }
    $INC{'A.pm'}  = 1
}

our $cond;

if ( $cond ) {
    print "Cond 1\n";
    eval {
        use A;
    }
}

emits

Begin 1
Importing A

So are you sure its fork that is the problem? What does it do if you don't fork?

Omitting F:SD:PDD will of course make any problems that relate to F:SD:PDD being unhappy with its ENV causing a problem ( and there are many specific documented requirements of an ENV for F:SD:PDD to work )

[basiliscos]

Hello Kent!

Thank you, I forget about distinction between compilation and execution phases in perl.

So, a bit simplified troublesome example is:

use 5.12.0;
use strict;
use warnings;

use Test::More;
use File::ShareDir::ProjectDistDir;

my $pid = fork();
if($pid == 0) {
    sleep 1;
    exit(0);
}
waitpid($pid, 0);
ok !$@;

done_testing;

Output:

c:\basiliscos\perl-watcher-master>prove  t\1.t
t\1.t ..
This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
t\1.t .. Dubious, test returned 3 (wstat 768, 0x300)
No subtests run

Test Summary Report
-------------------
t\1.t (Wstat: 768 Tests: 0 Failed: 0)
  Non-zero exit status: 3
  Parse errors: No plan found in TAP output
Files=1, Tests=0,  3 wallclock secs ( 0.09 usr +  0.08 sys =  0.17 CPU)
Result: FAIL

Yes, the problem seems to be in F:SD:PDD. If I use F:SD:PDD in my project test and do not fork, everything works fine (test: https://github.com/basiliscos/perl-watcher/blob/master/t/002-bootstrap.t , source: https://github.com/basiliscos/perl-watcher/blob/master/lib/App/PerlWatcher/Util/Bootstrap.pm , cpan-testers results: http://matrix.cpantesters.org/?dist=App-PerlWatcher-Engine%200.18;os=mswin32;perl=5.18.1;reports=1, http://www.cpantesters.org/cpan/report/4b16415b-6ff2-1014-8734-b10f2b825c07 )

If I fork, and do not use F:SD:PDD, everything also works fine (see https://github.com/basiliscos/perl-watcher/blob/master/t/015-ping-watcher.t , where fork is done indirectly via Test::TCP).

But use F:SD:PDD and then fork causes troubles on Win32, and I managed to reduce the problem to the example, provided above.

[kentfredric]

Hah. Weird. Do you get the same problem if you avoid calling import()? ie:

use File::ShareDir::ProjectDistDir ();

Sorry If I'm not more helpful, diagnosing Win32 bugs without Win32 is quite a challenge :/

[basiliscos]

Yes, avoiding calling import() causes no problem on fork()

[kentfredric]

Ok, my initial suspicion is there's something failing during exit, something to do with the code-refs hitting global destruction and going crazy, but you can help me rule out this possibility.

use 5.12.0;
use strict;
use warnings;

use Carp::Always;
use File::ShareDir::ProjectDistDir;

my $pid = fork();
if($pid == 0) {
    print "Message from $$\n";
    sleep 1;
    exit(0);
}
print "Message from master $$ pre-wait\n";
waitpid($pid, 0);
print "Message from master $$ post-wait\n";
print "Error is: $@";

Would you be able to run that with perl t/somescript.t and let me know what it produces?

The reason I've taken the Test::More stuff out is because sometimes, that can obscure useful failure context, and also added Carp::Always in the hope it gives a backtrace somewhere of the fail point.

And the prints are to let me see if it fails before, or after forking.

If it happens after forking, then its a global-destruction issue of some kind.

[basiliscos]

It seems that it fails IN fork. I have modified a bit an example:

use 5.12.0;
use strict;
use warnings;

use Carp::Always;
use File::ShareDir::ProjectDistDir;

print "Message from master $$ pre-fork\n";
my $pid = fork();
if($pid == 0) {
    print "Message from $$\n";
    sleep 1;
    exit(0);
}
print "Message from master $$ pre-wait\n";
waitpid($pid, 0);
print "Message from master $$ post-wait\n";
print "Error is: $@";

The output is:

c:\basiliscos\perl-watcher-master>perl t/2.t
Message from master 1416 pre-fork

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

c:\basiliscos\perl-watcher-master>

[kentfredric]

Ok, sorry for the delay, I had to acquire the help of people who understood this problem better. Seems when digging, the fault is triggered here: https://metacpan.org/source/DAGOLDEN/Path-Tiny-0.048/lib/Path/Tiny.pm#L39

sub CLONE { require threads; $TID = threads->tid }

CLONE is invoked during fork(), and it appears require threads, mid fork, is not the greatest idea.

See perldoc perlmod for details on CLONE

We still haven't worked out if this is a core bug or not that it fails in this way, and perhaps Path::Tiny could be fixed to avoid this problem somehow.

In the interim, there appears to be a trick that works.

 use Carp::Always;
 use File::ShareDir::ProjectDistDir;
+use threads;

Which at least avoids waiting till CLONE to load threads.pm and avoiding the bug circumstance.

Maybe, given that this bug only happens on Win32, and that you may not want a hard-dep on threads ( because many unix installs are built without threads, where on windows, forks are implemented in threads ), you may want something like

if ( $^O eq 'MSWin32' ) {
    require threads;
}

Somewhere before the fork to prevent problems.

I don't think its a long-term solution though, and I'll be following up on Path::Tiny to see if we can get a more long-term fix.

[kentfredric]

Re, changes in Path::Tiny, installing Path::Tiny >= 0.049 should also resolve your bug.

https://metacpan.org/source/DAGOLDEN/Path-Tiny-0.049/Changes#L7

https://metacpan.org/diff/release/DAGOLDEN/Path-Tiny-0.048/DAGOLDEN/Path-Tiny-0.049

So I'll consider this bug closed. :smiley:

[basiliscos]

Yep! Thanks a lot! That's was cool!