every OS child procs hangs forever if there is a blocked flock() in a win32 psuedofork

p5pRT commented 9 years ago

Migrated from rt.perl.org#123402 (status was 'open')

Searchable as RT123402$

p5pRT commented 9 years ago

From @bulk88

based on work in https://github.com/ingydotnet/inline-c-pm/pull/31

on HP-UX (any posix perl is good enough for the example)

~$perl hang.pl
core delete-me2 hang.pl refreshperl
delete-me delete-me3 perl the_lock
~$core delete-me2 hang.pl refreshperl
delete-me delete-me3 perl the_lock

~$perl -v

This is perl, v5.8.3 built for PA-RISC1.1-thread-multi
(with 8 registered patches, see perl -V for more detail)

on Win32 with psuedofork 5.21.6

C:\sources\hang>perl hang.pl
Volume in drive C has no label.
Volume Serial Number is 4C2D-0BFA

Directory of C:\sources\hang

12/09/2014 11:49 PM <DIR> .
12/09/2014 11:49 PM <DIR> ..
12/09/2014 11:45 PM 404 hang.pl
12/09/2014 11:50 PM 0 the_lock
2 File(s) 404 bytes
2 Dir(s) 1,918,545,920 bytes free
*****hanged forever******

This is because CloseHandle or kernel mode equivelent during process exit resource cleanup is a blocking call that prevents the process object from exiting untill all its resources are freed. "dir" AKA cmd.exe inherited the handle in the flock from the parent OS process's child psudofork (note the "dir" was launched from the parent psuedoprocess). Adding "$^F = -1;" before the fork on Win32 Perl is of no help (the default is "2" on Win32 Perl (and all Perls I think). So $^F is either broken or NA (Im not sure what its supposed to exactly do in pp_sys.c), or needs to be implemented for Win32 perl. Also in code I dont see any uses of FD_CLOEXEC compiled in but I'm not sure if its because of "#if defined(HAS_FCNTL) && defined(F_SETFD)" or outer enclosing CPP macros. Is Win32 Perl supposed to support (in libc or emulated in win32/win32.c like alot of other posix things) FD_CLOEXEC and $^F? (assuming $^F does what I think its supposed to do).

This bug did/does affect a real world CPAN module so its marked medium priority.

related reading http://www.agent31.eu/2007/01/perl-windows-and-file-locking.html http://blogs.msdn.com/b/oldnewthing/archive/2011/12/16/10248328.aspx but EXTENDED_STARTUPINFO_PRESENT isn't supported in XP/2003, so perl would have to serialize in perlhost.h/PERL_IMPLICIT_SYS, and remove then readd the inherit flag to libc file handles (good enough, rather than using debugging apis to get a list of all file handles in the process from NT kernel) with http://msdn.microsoft.com/en-us/library/windows/desktop/ms724935%28v=vs.85%29.aspx SetHandleInformation , if you think about it, isn't SetHandleInformation and HANDLE_FLAG_INHERIT the Win32 equivelent to FD_CLOEXEC?

So what is the plan for this bug? comments from p5p?

Perl Info

``` Flags: category=core severity=medium Site configuration information for perl 5.21.6: Configured by Owner at Thu Nov 13 18:20:06 2014. Summary of my perl5 (revision 5 version 21 subversion 6) configuration: Derived from: 5e1acb3ae8d10f0a0404ec4d30afb925b08b53fd Ancestor: b255a112693312907754193cdbc1648d9182cb0d Platform: osname=MSWin32, osvers=5.1, archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended, useposix=true, d_sigaction=undef useithreads=define, usemultiplicity=define use64bitint=undef, use64bitall=undef, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cl', ccflags ='-nologo -GF -W3 -Od -MD -Zi -DDEBUGGING -G7 -GL -DWIN32 -D_CONSOLE -DNO_STRICT -DPERL_TEXTMODE_SCRIPTS -DPERL_HASH_FUNC_ONE_AT_A_TIME -DNO_MATHOMS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -D_USE_32BIT_TIME_T', optimize='-Od -MD -Zi -DDEBUGGING -G7 -GL', cppflags='-DWIN32' ccversion='13.10.6030', gccversion='', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234, doublekind=3 d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=8, longdblkind=0 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -libpath:"c:\perl521\lib\CORE" -machine:x86' libpth="C:\Program Files\Microsoft Visual Studio .NET 2003\VC7\lib" libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl521.lib gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -libpath:"c:\perl521\lib\CORE" -machine:x86' Locally applied patches: uncommitted-changes 02b4b73f6c4578c66ae7106592964b8853d4db30 45e05eceb42cd889a80ef0875887fcb62181cbb6 225b72a0c1e8cf621f10ed445e0260410082bfc6 7f871da7cac3993a56343b032f9d7eff91422224 0b68901531d0a01506365798f883b5f08b1adb89 a87204c1df9892bf05451407ab107a76ece4810b 5e1acb3ae8d10f0a0404ec4d30afb925b08b53fd @INC for perl 5.21.6: C:/perl521/site/lib C:/perl521/lib . Environment for perl 5.21.6: HOME (unset) LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=C:\perl521\bin;C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\IDE;C:\Program Files\Microsoft Visual Studio .NET 2003\VC7\BIN;C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools;C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools\bin\prerelease;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\system32\wbem; PERL_BADLANG (unset) SHELL (unset) ```

p5pRT commented 9 years ago

From @bulk88

hang.pl

use Fcntl ':flock';
my $file = 'the_lock';

$^F = -1;
if(fork == 0){
    sleep 1;
    open my $lockfh, '>', $file or die "lockfile $file: $!";
    flock($lockfh, LOCK_EX) or die "flock: $!\n";
    flock($lockfh, LOCK_UN);
}
open my $lockfh, '>', $file or die "lockfile $file: $!";
flock($lockfh, LOCK_EX) or die "flock: $!\n";
sleep 2;
system($^O eq 'MSWin32' ? 'dir' : 'ls');
flock($lockfh, LOCK_UN);

p5pRT commented 9 years ago

From @tonycoz

This doesn't hang on Windows 7 x64.

Tony

p5pRT commented 9 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 9 years ago

From @bulk88

Hangs on Server 2003 R2 SP2 x64 and XP SP3 x86 for me. Ill make a guess and say it hangs on \< NT 6.0 kernel. -- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 9 years ago

From @bulk88

-- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 9 years ago

From @bulk88

I tested hang.pl on my Win7 32 bit. It didn't hang. That confirms its NT \<6 bug.

-- bulk88 ~ bulk88 at hotmail.com

Perl / perl5