Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.85k stars 527 forks source link

exit()/exec() crashes Perl after fork #6338

Open p5pRT opened 21 years ago

p5pRT commented 21 years ago

Migrated from rt.perl.org#21382 (status was 'open')

Searchable as RT21382$

p5pRT commented 21 years ago

From Jens.Lippmann@t-mobile.de

Created by jens.lippmann@t-mobile.de

Hello!

I'm into Perl for more than 8 years and I cannot overemphasize that I appreciated every single minute programming with it.

Now\, I actually spent quite a large amount of time to find a nasty bug that crashes Perl on Windows NT when I call fork and exec. The problem is not related to any specific Perl but rather sticks to Perl ports that create threads instead of processes. As soon as the exec'd task terminates there is a considerable high chance (obviously a race condition) to crash perl. When I replace exec() with system() and exit() the problem pertains\, but not if I use system() and POSIX​::_exit().

I am sending this bug report to the mainstream Perl development instead of any specific Win32 support group such as ActiveState or Indigo just because the 'bug' I am referring herein seems to be related in a deficient Perl documentation on this topic.

My pledge is to build up a FAQ topic (perldoc -q) for fork that helps out of this remedy\, and to extend the fork/exec sections in the perlfuncs doc module\, and to overhaul the notorious 'if ($pid = fork()) { ... } else { ... exec/exit; }' examples that are used throughout the perlipc doc.

It is generally a bad idea to use exit() after fork()\, unless you want to create a daemon process (see http​://www.erlenstar.demon.co.uk/unix/faq_toc.html "1.1.3 Why use_exit rather than exit in the child branch of a fork?"). It is also a bad idea to use exec() after fork() on a Win32 port for the simple reason that exec() itself performs cleanup action on process termination similar to exit().

It is no good way to leave the user undocumented and let the developers run into the same obstacles again with every new release. Meanwhile there are half a dozen Win32 module extensions on CPAN\, but none seems to come with support throughout all Win32 platforms or availability for any perl distribution. For the sake of transparency and portability\, why can't Perl itself solve the exit handling to smooth out thread and non-thread forks? I recommend to either - keep the transparency of fork() but then make exit() transparent as well   (i.e. remember the 'thread' state and act accordingly)\, or - do not support fork() on Win32 but vfork() and provide a function like   vexit()

I further recommend to generally incorporate POSIX​::_exit() into the Perl core and to add a hint about the batch command 'start' which can be used to conveniently spawn a task under Win32.

kind regards Jens Lippmann

Perl Info ``` Flags: category=docs severity=medium Site configuration information for perl v5.6.1: Configured by Indy at Sun May 6 21:35:42 2001. Summary of my perl5 (revision 5 version 6 subversion 1) configuration: Platform: osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended, useposix=true, d_sigaction=undef usethreads=undef use5005threads=undef useithreads=define usemultiplicity=define useperlio=undef d_sfio=undef uselargefiles=undef usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef Compiler: cc='cl', ccflags ='-nologo -O1 -MD -DNDEBUG -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DPERL_MSVCRT_READFIX', optimize='-O1 -MD -DNDEBUG', cppflags='-DWIN32' ccversion='', gccversion='', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=4 alignbytes=8, usemymalloc=n, prototype=define Linker and Libraries: ld='', ldflags ='-nologo -nodefaultlib -release -libpath:"f:\dev\dp\perl-5.6_install\lib\CORE" -machine:x86' libpth=\lib libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib msvcrt.lib perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib msvcrt.lib libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl56.lib Dynamic Linking: dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -release -libpath:"f:\dev\dp\perl-5.6_install\lib\CORE" -machine:x86' Locally applied patches: ACTIVEPERL_LOCAL_PATCHES_ENTRY @INC for perl v5.6.1: C:/PROGS/perl/lib C:/PROGS/perl/site/lib . Environment for perl v5.6.1: HOME (unset) LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=c:\Progs\Rational\ClearCase\bin;W:\TOOLS\CLEARMOBIL\bin;C:\WINNT\system32;C:\WINNT;C:\PROGS\TEAMLINK;C:\PROGS\Rational\COMMON;C:\PROGS\VS\COMMON\Tools\WinNT;C:\PROGS\VS\COMMON\MSDev\Bin;C:\PROGS\VS\COMMON\Tools;c:\progs\vs\vc\bin;C:\PROGS\MAESTRO;;C:\PROGS\Office\Office;E:\rtools;O:\wna_source\wna_apps\src\regtest\tools;O:\wna_source\wna_apps\src\helper\batch;O:\wna_source\wna_apps\src\onc-rpc-nt\bin;R:\oracle\816\bin;O:\tps_source\cyg\src\cygwin\bin;O:\private_binary\bin\nt\Debug;O:\private_binary\bin\nt\Release;O:\car_binary\bin\nt\debug;O:\car_binary\bin\nt\release;O:\tps_binary\bin\nt\debug;O:\tps_binary\bin\nt\release;O:\tfs_binary\bin\nt\debug;O:\tfs_binary\bin\nt\release;L:\Carmen.ABN\5.0\common;C:\progs\xmlpad;C:\progs\textpad;C:\progs\perl\bin PERL_BADLANG (unset) SHELL (unset) ```
p5pRT commented 21 years ago

arthur@contiller.se - Status changed from 'new' to 'open'

p5pRT commented 21 years ago

From @cwest

[Jens.Lippmann \<!--c--> \at\ \<!--a--> t-mobile.de - Wed Feb 26 13​:26​:38 2003]​:

This is a bug report for perl from jens.lippmann \<!--c--> \at\ \<!--a--> t- mobile.de\, generated with the help of perlbug 1.33 running under perl v5.6.1.

----------------------------------------------------------------- [Please enter your report here] Hello!

I'm into Perl for more than 8 years and I cannot overemphasize that I appreciated every single minute programming with it.

Excellent!

Now\, I actually spent quite a large amount of time to find a nasty bug that crashes Perl on Windows NT when I call fork and exec. The problem is not related to any specific Perl but rather sticks to Perl ports that create threads instead of processes. As soon as the exec'd task terminates there is a considerable high chance (obviously a race condition) to crash perl. When I replace exec() with system() and exit() the problem pertains\, but not if I use system() and POSIX​::_exit().

I am sending this bug report to the mainstream Perl development instead of any specific Win32 support group such as ActiveState or Indigo just because the 'bug' I am referring herein seems to be related in a deficient Perl documentation on this topic.

Sounds reasonable...

My pledge is to build up a FAQ topic (perldoc -q) for fork that helps out of this remedy\, and to extend the fork/exec sections in the perlfuncs doc module\, and to overhaul the notorious 'if ($pid = fork()) { ... } else { ... exec/exit; }' examples that are used throughout the perlipc doc.

How's this coming along?

It is generally a bad idea to use exit() after fork()\, unless you want to create a daemon process (see http​://www.erlenstar.demon.co.uk/unix/faq_toc.html "1.1.3 Why use_exit rather than exit in the child branch of a fork?"). It is also a bad idea to use exec() after fork() on a Win32 port for the simple reason that exec() itself performs cleanup action on process termination similar to exit().

It is no good way to leave the user undocumented and let the developers run into the same obstacles again with every new release. Meanwhile there are half a dozen Win32 module extensions on CPAN\, but none seems to come with support throughout all Win32 platforms or availability for any perl distribution. For the sake of transparency and portability\, why can't Perl itself solve the exit handling to smooth out thread and non-thread forks? I recommend to either - keep the transparency of fork() but then make exit() transparent as well (i.e. remember the 'thread' state and act accordingly)\, or - do not support fork() on Win32 but vfork() and provide a function like vexit()

I further recommend to generally incorporate POSIX​::_exit() into the Perl core and to add a hint about the batch command 'start' which can be used to conveniently spawn a task under Win32.

This isn't quite a documentation bug\, so lets work on the documentation bug first.
Please let me know where you are on this\, I'd like to get this bug resolved. Thanks.

  Casey West

p5pRT commented 21 years ago

From @cwest

[replying and posting to p5p]

[Jens.Lippmann \<!--c--> \at\ \<!--a--> t-mobile.de - Wed Feb 26 13​:26​:38 2003]​:

Now\, I actually spent quite a large amount of time to find a nasty bug that crashes Perl on Windows NT when I call fork and exec. The problem is not related to any specific Perl but rather sticks to Perl ports that create threads instead of processes. As soon as the exec'd task terminates there is a considerable high chance (obviously a race condition) to crash perl. When I replace exec() with system() and exit() the problem pertains\, but not if I use system() and POSIX​::_exit().

Ok\, I have no access to a Win32 machine so I can't go on testing this\, but I can believe you.

I am sending this bug report to the mainstream Perl development instead of any specific Win32 support group such as ActiveState or Indigo just because the 'bug' I am referring herein seems to be related in a deficient Perl documentation on this topic.

My pledge is to build up a FAQ topic (perldoc -q) for fork that helps out of this remedy\, and to extend the fork/exec sections in the perlfuncs doc module\, and to overhaul the notorious 'if ($pid = fork()) { ... } else { ... exec/exit; }' examples that are used throughout the perlipc doc.

None of this has been done yet. Is it all that necessary? Has anyone else found the need to have this documented in some way\, somewhere?

p5pRT commented 21 years ago

From Jens.Lippmann@t-mobile.de

My pledge is to build up a FAQ topic (perldoc -q) for fork that helps out of this remedy\, and to extend the fork/exec sections in the perlfuncs doc module\, and to overhaul the notorious 'if ($pid = fork()) { ... } else { ... exec/exit; }' examples that are used throughout the perlipc doc.

None of this has been done yet. Is it all that necessary? Has anyone else found the need to have this documented in some way\, somewhere?

I'm currently building up an extension to perlfaq8.pod top 'How do I start a process in the background?'.

Later I'll try to narrow down my script in size to a sniplet that is able to reproduce the crash. Although it might be troublesome since the crash (race condition) seems to be related to the size of the process image.

kind regards Jens

p5pRT commented 21 years ago

From sky@nanisky.com

On torsdag\, maj 15\, 2003\, at 18​:50 Europe/Stockholm\, Lippmann Jens\,E2 wrote​:

Later I'll try to narrow down my script in size to a sniplet that is able to reproduce the crash. Although it might be troublesome since the crash (race condition) seems to be related to the size of the process image.

kind regards Jens

I would think it is more likely related to a module you are using that doesn't cope well with the psuedo fork.

Can we have a backtrace?

Arthur

p5pRT commented 21 years ago

From Jens.Lippmann@t-mobile.de

Hi

I cannot shrink the script\, as soon as I remove a certain quantity of text the crash disappears. Well I admit that I use file IO quite extensively and keep file handles in memory\, so that both parent and child refer to the same descriptors. BUT the child does not make use of those descriptors\, it simply calls 'system("echo test") && exit' Immediately after the child exits Perl will crash.

I'm in an urgent need of a Perl Debug-Build because I can't afford the time to play catch-me with this bug. Any Perl that runs on Win32 and uses threads is welcome. My crash occurs with Indigo 5.6.1/5.8.0 as well as ActiveState 5.6.1. If someone can point me to a Win32 Perl development distribution I would be *greatly* relieved.

-----Ursprüngliche Nachricht----- Von​: Lippmann Jens\,E2 Gesendet am​: Donnerstag\, 15. Mai 2003 18​:50 An​: 'perlbug-followup@​perl.org' Betreff​: AW​: [perl #21382] exit()/exec() crashes Perl after fork

My pledge is to build up a FAQ topic (perldoc -q) for fork that helps out of this remedy\, and to extend the fork/exec sections in the perlfuncs doc module\, and to overhaul the notorious 'if ($pid = fork()) { ... } else { ... exec/exit; }' examples that are used throughout the perlipc doc.

None of this has been done yet. Is it all that necessary? Has anyone else found the need to have this documented in some way\, somewhere?

I'm currently building up an extension to perlfaq8.pod top 'How do I start a process in the background?'.

Later I'll try to narrow down my script in size to a sniplet that is able to reproduce the crash. Although it might be troublesome since the crash (race condition) seems to be related to the size of the process image.

kind regards Jens

p5pRT commented 10 years ago

From @bulk88

To move this old ticket along

On Sat May 24 00​:40​:29 2003\, Jens.Lippmann@​t-mobile.de wrote​:

Hi

I cannot shrink the script\, as soon as I remove a certain quantity of text the crash disappears. Well I admit that I use file IO quite extensively and keep file handles in memory\, so that both parent and child refer to the same descriptors. BUT the child does not make use of those descriptors\, it simply calls 'system("echo test") && exit' Immediately after the child exits Perl will crash.

I'm in an urgent need of a Perl Debug-Build because I can't afford the time to play catch-me with this bug. Any Perl that runs on Win32 and uses threads is welcome. My crash occurs with Indigo 5.6.1/5.8.0 as well as ActiveState 5.6.1. If someone can point me to a Win32 Perl development distribution I would be *greatly* relieved.

-----Ursprüngliche Nachricht----- Von​: Lippmann Jens\,E2 Gesendet am​: Donnerstag\, 15. Mai 2003 18​:50 An​: 'perlbug-followup@​perl.org' Betreff​: AW​: [perl #21382] exit()/exec() crashes Perl after fork

My pledge is to build up a FAQ topic (perldoc -q) for fork that helps out of this remedy\, and to extend the fork/exec sections in the perlfuncs doc module\, and to overhaul the notorious 'if ($pid = fork()) { ... } else { ... exec/exit; }' examples that are used throughout the perlipc doc.

None of this has been done yet. Is it all that necessary? Has anyone else found the need to have this documented in some way\, somewhere?

I'm currently building up an extension to perlfaq8.pod top 'How do I start a process in the background?'.

I dont think this patch will ever come since it is 2013. The text\, in question\, now has a disclaimer that fork/exec aren't the only way to background run.


=head2 How do I start a process in the background?

(contributed by brian d foy)

There's not a single way to run code in the background so you don't have to wait for it to finish before your program moves on to other tasks. Process management depends on your particular operating system\, and many of the techniques are in L\.

Several CPAN modules may be able to help\, including C\<IPC​::Open2> or C\<IPC​::Open3>\, C\<IPC​::Run>\, C\<Parallel​::Jobs>\, C\<Parallel​::ForkManager>\, C\\, C\<Proc​::Background>\, and C\<Win32​::Process>. There are many other modules you might use\, so check those namespaces for other options too.


Later I'll try to narrow down my script in size to a sniplet that is able to reproduce the crash. Although it might be troublesome since the crash (race condition) seems to be related to the size of the process image.

Is this a crash with 5.005 threads?

-- bulk88 ~ bulk88 at hotmail.com

toddr commented 4 years ago

@bulk88 the original report was on 5.6.1