From frank.wang@validusresearch.com on 2011-07-27 21:41:15
:
Hi Mr. Mueller,
When using pp to create a stanalone executable in activeperl for x64, if the source code used Parallel::Loops, the executable will crash and the following is the problem detail:
Problem signature:
Problem Event Name: APPCRASH
Application Name: pre-process-elt-x64.exe
Application Version: 0.0.0.0
Application Timestamp: 4e3026dd
Fault Module Name: perl512.dll
Fault Module Version: 5.12.2.1203
Fault Module Timestamp: 4d009fe0
Exception Code: c0000005
Exception Offset: 000000000013eb35
OS Version: 6.0.6002.2.2.0.18.10
Locale ID: 1033
Additional Information 1: 98c4
Additional Information 2: adc3265ea6226a0345ffdbdc4025017e
Additional Information 3: 360c
Additional Information 4: f2486cf67a3bbd6d9170c007387206e1
Is this problem related to how fork in perl is handled in windows or is this problem only related to Parallel::Loops? But the perl code itself works without any problem.
Thank you very much for your attention.
Frank Wang
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
From rschupp@cpan.org on 2011-07-27 22:03:35
:
On 2011-07-27 17:41:15, frank.wang@validusresearch.com wrote:
> When using pp to create a stanalone executable in activeperl for x64,
> if the source code used Parallel::Loops, the executable will crash and
> the following is the problem detail:
This information is useless.
Please provide a minimal code example that exhibits the problem.
Also include the output of "perl -V" and the versions of
PAR and PAR::Packer used.
> But the perl code itself works without any problem.
By that you mean: the code works when not packed?
Cheers, Roderich
From frank.wang@validusresearch.com on 2011-07-28 13:43:16
:
Hi Roderich,
Yes the code work fine and fast when not packed. Only crash when run packed code.
The following is a minimal code example that exhibits the problem:
#!/usr/bin/perl
use strict;
use warnings;
use diagnostics;
$|=1;
my $workers=@ARGV ? shift : 16;
use Parallel::Loops;
my @modelfiles=("ct_125_gulfstream_ws_26119a.txt");
my $pl = Parallel::Loops->new($workers);
$pl->foreach(\@modelfiles,
sub {
my $mf = $_;
print $mf;
}
);
And this is the output of perl-V:
Set up gcc environment - 4.4.5 20101001 (release) [svn/rev.164871 - mingw-w64/oz]
Summary of my perl5 (revision 5 version 12 subversion 2) configuration:
Platform:
osname=MSWin32, osvers=5.2, archname=MSWin32-x64-multi-thread
uname=''
config_args='undef'
hint=recommended, useposix=true, d_sigaction=undef
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=define, use64bitall=undef, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='gcc', ccflags =' -DNDEBUG -O2 -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DCONSERVATIVE -DUSE_SITECUSTOMIZE -DPRIVLIB_LAST_IN_INC -DPERL_IMPLI
CIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX -DWIN64',
optimize='-O2',
cppflags='-DWIN32'
ccversion='', gccversion='4.4.5 20101001 (release) [svn/rev.164871 - mingw-w64/oz]', gccosandvers=''
intsize=4, longsize=4, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=8
ivtype='__int64', ivsize=8, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='g++', ldflags ='-s -L"C:\Perl64\lib\CORE"'
libpth="C:\mingw64\lib" "C:\mingw64\x86_64-w64-mingw32\lib"
libs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lve
rsion -lodbc32 -lodbccp32 -lcomctl32 -lmsvcrt
perllibs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm
-lversion -lodbc32 -lodbccp32 -lcomctl32 -lmsvcrt
libc=-lmsvcrt, so=dll, useshrplib=true, libperl=libperl512.a
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags='-s -mdll -L"C:\Perl64\lib\CORE"'
Characteristics of this binary (from libperl):
Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV
PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS
PERL_MALLOC_WRAP PL_OP_SLAB_ALLOC USE_64_BIT_INT
USE_ITHREADS USE_LARGE_FILES USE_PERLIO USE_PERL_ATOF
USE_SITECUSTOMIZE
Locally applied patches:
ActivePerl Build 1203 [294165]
1fd8fa4 Add Wolfram Humann to AUTHORS
f120055 make string-append on win32 100 times faster
a2a8d15 Define _USE_32BIT_TIME_T for VC6 and VC7
007cfe1 Don't pretend to support really old VC++ compilers
6d8f7c9 Get rid of obsolete PerlCRT.dll support
d956618 Make Term::ReadLine::findConsole fall back to STDIN if /dev/tty can't be opened
321e50c Escape patch strings before embedding them in patchlevel.h
Built under MSWin32
Compiled at Dec 9 2010 00:50:22
%ENV:
PERL5OPT="-MConfig_w64"
@INC:
C:/Perl64/site/lib
C:/Perl64/lib
.
The file " C:\Perl64\site\lib\PAR \ Packer .pm" has
our $VERSION = '1.010';
Actually my previous version of Par::Packer is 1.008 and showed this problem so I updated it to 1.010 and hope this problem will go away but seems it is not.
(Also if we do not use parallel loop the packed code run fine.)
Thank you.
Frank
-----Original Message-----
From: Roderich Schupp via RT [mailto:bug-PAR-Packer@rt.cpan.org]
Sent: Wednesday, July 27, 2011 6:04 PM
To: Wang, Frank
Subject: [rt.cpan.org #69848] pp created exe crash if useing parallel loops
<URL: https://rt.cpan.org/Ticket/Display.html?id=69848 >
On 2011-07-27 17:41:15, frank.wang@validusresearch.com wrote:
> When using pp to create a stanalone executable in activeperl for x64,
> if the source code used Parallel::Loops, the executable will crash and
> the following is the problem detail:
This information is useless.
Please provide a minimal code example that exhibits the problem.
Also include the output of "perl -V" and the versions of PAR and PAR::Packer used.
> But the perl code itself works without any problem.
By that you mean: the code works when not packed?
Cheers, Roderich
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email ______________________________________________________________________
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
From rschupp@cpan.org on 2011-08-01 14:26:13
:
On 2011-07-28 09:43:16, frank.wang@validusresearch.com wrote:
> The following is a minimal code example that exhibits the problem:
Thanks. The good news is that I can easily reproduce the
problem here (with e.g. ActiveState 5.10.1 on 32-bit Windows XP).
The bad news is that I have no clue what's going on.
Looks like the problem occurs in some END block or in the
global destruction phase. Note that a packed executable actually
runs in a custom Perl interpreter which might have some
problem with the fork emulation on Windows.
I keep looking into this, but don't hold your breadth.
Cheers, Roderich
>
> #!/usr/bin/perl
> use strict;
> use warnings;
> use diagnostics;
> $|=1;
> my $workers=@ARGV ? shift : 16;
> use Parallel::Loops;
> my @modelfiles=("ct_125_gulfstream_ws_26119a.txt");
> my $pl = Parallel::Loops->new($workers);
> $pl->foreach(\@modelfiles,
> sub {
> my $mf = $_;
> print $mf;
> }
> );
>
> And this is the output of perl-V:
>
> Set up gcc environment - 4.4.5 20101001 (release) [svn/rev.164871 -
> mingw-w64/oz]
> Summary of my perl5 (revision 5 version 12 subversion 2)
> configuration:
>
> Platform:
> osname=MSWin32, osvers=5.2, archname=MSWin32-x64-multi-thread
> uname=''
> config_args='undef'
> hint=recommended, useposix=true, d_sigaction=undef
> useithreads=define, usemultiplicity=define
> useperlio=define, d_sfio=undef, uselargefiles=define,
> usesocks=undef
> use64bitint=define, use64bitall=undef, uselongdouble=undef
> usemymalloc=n, bincompat5005=undef
> Compiler:
> cc='gcc', ccflags =' -DNDEBUG -O2 -DWIN32 -D_CONSOLE -DNO_STRICT
> -DHAVE_DES_FCRYPT -DCONSERVATIVE -DUSE_SITECUSTOMIZE
> -DPRIVLIB_LAST_IN_INC -DPERL_IMPLI
> CIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX
> -DWIN64',
> optimize='-O2',
> cppflags='-DWIN32'
> ccversion='', gccversion='4.4.5 20101001 (release) [svn/rev.164871
> - mingw-w64/oz]', gccosandvers=''
> intsize=4, longsize=4, ptrsize=8, doublesize=8, byteorder=12345678
> d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=8
> ivtype='__int64', ivsize=8, nvtype='double', nvsize=8,
> Off_t='__int64', lseeksize=8
> alignbytes=8, prototype=define
> Linker and Libraries:
> ld='g++', ldflags ='-s -L"C:\Perl64\lib\CORE"'
> libpth="C:\mingw64\lib" "C:\mingw64\x86_64-w64-mingw32\lib"
> libs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32
> -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32
> -lmpr -lwinmm -lve
> rsion -lodbc32 -lodbccp32 -lcomctl32 -lmsvcrt
> perllibs= -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool
> -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid
> -lws2_32 -lmpr -lwinmm
> -lversion -lodbc32 -lodbccp32 -lcomctl32 -lmsvcrt
> libc=-lmsvcrt, so=dll, useshrplib=true, libperl=libperl512.a
> gnulibc_version=''
> Dynamic Linking:
> dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
> cccdlflags=' ', lddlflags='-s -mdll -L"C:\Perl64\lib\CORE"'
>
>
> Characteristics of this binary (from libperl):
> Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV
> PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS
> PERL_MALLOC_WRAP PL_OP_SLAB_ALLOC
> USE_64_BIT_INT
> USE_ITHREADS USE_LARGE_FILES USE_PERLIO
> USE_PERL_ATOF
> USE_SITECUSTOMIZE
> Locally applied patches:
> ActivePerl Build 1203 [294165]
> 1fd8fa4 Add Wolfram Humann to AUTHORS
> f120055 make string-append on win32 100 times faster
> a2a8d15 Define _USE_32BIT_TIME_T for VC6 and VC7
> 007cfe1 Don't pretend to support really old VC++ compilers
> 6d8f7c9 Get rid of obsolete PerlCRT.dll support
> d956618 Make Term::ReadLine::findConsole fall back to STDIN if
> /dev/tty can't be opened
> 321e50c Escape patch strings before embedding them in
> patchlevel.h
> Built under MSWin32
> Compiled at Dec 9 2010 00:50:22
> %ENV:
> PERL5OPT="-MConfig_w64"
> @INC:
> C:/Perl64/site/lib
> C:/Perl64/lib
> .
>
>
> The file " C:\Perl64\site\lib\PAR \ Packer .pm" has
> our $VERSION = '1.010';
>
> Actually my previous version of Par::Packer is 1.008 and showed this
> problem so I updated it to 1.010 and hope this problem will go away
> but seems it is not.
> (Also if we do not use parallel loop the packed code run fine.)
>
> Thank you.
>
> Frank
>
>
> -----Original Message-----
> From: Roderich Schupp via RT [mailto:bug-PAR-Packer@rt.cpan.org]
> Sent: Wednesday, July 27, 2011 6:04 PM
> To: Wang, Frank
> Subject: [rt.cpan.org #69848] pp created exe crash if useing parallel
> loops
>
> <URL: https://rt.cpan.org/Ticket/Display.html?id=69848 >
>
> On 2011-07-27 17:41:15, frank.wang@validusresearch.com wrote:
> > When using pp to create a stanalone executable in activeperl for
> x64,
> > if the source code used Parallel::Loops, the executable will crash
> and
> > the following is the problem detail:
>
> This information is useless.
>
> Please provide a minimal code example that exhibits the problem.
> Also include the output of "perl -V" and the versions of PAR and
> PAR::Packer used.
>
> > But the perl code itself works without any problem.
>
> By that you mean: the code works when not packed?
>
> Cheers, Roderich
>
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email
> ______________________________________________________________________
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email
> ______________________________________________________________________
From rschupp@cpan.org on 2011-08-02 09:26:22
:
On 2011-08-01 10:26:13, RSCHUPP wrote:
> I keep looking into this,
The actual problem is that a packed script actually runs inside
a BEGIN block. If you simply put everything in your example script
(except the "use" statements) into a BEGIN block, then it
will blow up even unpacked.
The perlfork man page explicitly states that fork (i.e. the
emulation for Windows) might fail when invoked inside BEGIN.
As to why PAR::Packer wraps a packed script into a BEGIN block:
I don't know - it has always been the case (according to the
history in Subversion).
Cheers, Roderich
From frank.wang@validusresearch.com on 2011-08-02 21:55:58
:
Hi Rodrich,
Thank you for inform me of this finding. Just wondering maybe it is possible to
update PAR::Packer soon since wrap packed script into a BEGIN block or into BODY
block seems do not need much change? and the pack function and parallel process
function are very desirable feature of Perl.
Regards,
Frank
-----Original Message-----
From: Roderich Schupp via RT [mailto:bug-PAR-Packer@rt.cpan.org]
Sent: Tuesday, August 02, 2011 5:26 AM
To: Wang, Frank
Subject: [rt.cpan.org #69848] pp created exe crash if useing parallel loops
<URL: https://rt.cpan.org/Ticket/Display.html?id=69848 >
On 2011-08-01 10:26:13, RSCHUPP wrote:
> I keep looking into this,
The actual problem is that a packed script actually runs inside
a BEGIN block. If you simply put everything in your example script
(except the "use" statements) into a BEGIN block, then it
will blow up even unpacked.
The perlfork man page explicitly states that fork (i.e. the
emulation for Windows) might fail when invoked inside BEGIN.
As to why PAR::Packer wraps a packed script into a BEGIN block:
I don't know - it has always been the case (according to the
history in Subversion).
Cheers, Roderich
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
______________________________________________________________________
From rschupp@cpan.org on 2011-08-03 06:38:43
:
On 2011-08-02 17:55:58, frank.wang@validusresearch.com wrote:
> Just wondering maybe it is possible to
> update PAR::Packer soon since wrap packed script into a BEGIN block or
> into BODY block seems do not need much change?
Sorry, unlikely soon. There has to be a reason why practically all
of script/par.pl (which eventually calls "do 'your_original_script'")
is wrapped in BEGIN (and eval {} to boot), but I don't know it
(mind you, I'm not the original author). Any change in this crucial
piece of the PAR::Packer machinery needs consideration and
extensive testing for which I don't have time ATM.
Cheers, Roderich
From rschupp@cpan.org on 2011-08-09 10:52:49
:
On 2011-08-03 02:38:43, RSCHUPP wrote:
> piece of the PAR::Packer machinery needs consideration and
> extensive testing for which I don't have time ATM.
Just as a reminder to myself...
Things done to narrow down the problem - but still no solution:
- reduced the span of the BEGIN block in script/par.pl
- verified that the packed script is NOT executed from "inside"
BEGIN or eval() any more
- set PL_perl_destruct_level=1 in myldr/main.c for perl_construct()
and perl_destruct() call (but to no avail)
- problem occurs definitely AFTER the end of normal execution
(but still in perl_run())
- using Devel::Symdump we see that the only END blocks registered
are in File::Temp, PAR and par.pl
- by instrumenting these END blocks we see that the problem
occurs AFTER they have been run
- the problem is not specific to Parallel::Loops or
Parallel::ForkManager, the following script shows the same
symptoms (with or without the wait()ing at the end)
use strict;
use warnings;
for (1..5)
{
my $pid = fork();
die "fork failed: $!" unless defined $pid;
if ($pid == 0)
{
# child
print STDERR "child $$\n";
exit(0);
}
}
print STDERR "waiting for children ...\n";
while ((my $pid = wait()) != -1)
{
print STDERR "reaped $pid\n";
}
On 2020-04-22 11:22:03, vld.vld@sfr.fr wrote:
> > my $finished = waitpid -1, 0;
What about
my $finished = waitpid($pid, 0);
Please switch to English.
Cheers, Roderich
On 2020-04-23 12:09:26, vld.vld@sfr.fr wrote:
> is there a way to use PAR :: Packer with a module using fork like poe ::
> wheel :: run or Parallel :: ForkManager or simply by using the perl fork
> command.
Dunno. The difference in behaviour of your sample script in "normal"
and pp'ed versions is definitively a bug, but I have no means
(i.e. no Windows) to investigate it. In any case, fork and wait* on
Windows is a minefield.
You might also ask for help on the PAR mailing list, par@perl.org
(no registration required).
> Or pierce another way to compile a perl script in exe
What do you mean by "compile a perl script in exe"?
Cheeers, Roderich
Migrated from rt.cpan.org#69848 (status was 'open')
Requestors:
From frank.wang@validusresearch.com on 2011-07-27 21:41:15 :
From rschupp@cpan.org on 2011-07-27 22:03:35 :
From frank.wang@validusresearch.com on 2011-07-28 13:43:16 :
From rschupp@cpan.org on 2011-08-01 14:26:13 :
From rschupp@cpan.org on 2011-08-02 09:26:22 :
From frank.wang@validusresearch.com on 2011-08-02 21:55:58 :
From rschupp@cpan.org on 2011-08-03 06:38:43 :
From rschupp@cpan.org on 2011-08-09 10:52:49 :
From rschupp@cpan.org on 2020-04-22 13:14:05 :
From vld.vld@sfr.fr on 2020-04-22 15:22:03 :
From rschupp@cpan.org on 2020-04-22 15:49:39 :
From vld.vld@sfr.fr on 2020-04-22 16:04:49 :
From vld.vld@sfr.fr on 2020-04-23 16:09:26 :
From rschupp@cpan.org on 2020-04-25 11:00:18 :