search

Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.91k stars 542 forks source link

PerlIO::via isn't thread safe #8713

Open p5pRT opened 17 years ago

p5pRT commented 17 years ago

Migrated from rt.perl.org#41106 (status was 'open')

Searchable as RT41106$

p5pRT commented 17 years ago

From blgl@hagernas.com

Created by blgl@hagernas.com

This outputs "Hello=3DWorld!" as expected but then crashes.

### begin example program #! /usr/unstable/bin/perl

use strict; use warnings; use threads; use PerlIO​::via​::QuotedPrint;

open(my $foo\,'>&​:via(QuotedPrint)'\,*STDOUT)   or die "open​: $!"; print $foo "Hello=World!\n"   or die "print​: $!"; async {   print $foo "Hello=from=Thread!\n"; }->join(); ### end example program

### begin debugger log Program received signal EXC_BAD_ACCESS\, Could not access memory. Reason​: KERN_INVALID_ADDRESS at address​: 0xabababaf 0x0008f580 in Perl_gv_fetchpvn_flags (my_perl=0x1811400\,
nambeg=0xbfffe520 "UNIVERSAL​::"\, full_len=11\, flags=0\, sv_type=12) at
gv.c​:815 815 if (!stash || !SvREFCNT(stash)) /* symbol table
under destruction */ (gdb) print stash $1 = (HV *) 0xabababab (gdb) bt #0 0x0008f580 in Perl_gv_fetchpvn_flags (my_perl=0x1811400\,
nambeg=0xbfffe520 "UNIVERSAL​::"\, full_len=11\, flags=0\, sv_type=12) at
gv.c​:815 #1 0x0008ece0 in Perl_gv_stashpvn (my_perl=0x1811400\, name=0x2d3bf0
"UNIVERSAL"\, namelen=11\, create=0) at gv.c​:736 #2 0x0008c410 in Perl_gv_fetchmeth (my_perl=0x1811400\,
stash=0x182af64\, name=0x71cf34 "GETARG"\, len=6\, level=0) at gv.c​:410 #3 0x007167b8 in PerlIOVia_fetchmethod (my_perl=0x1811400\,
s=0x1137e20\, method=0x71cf34 "GETARG"\, save=0x1137e58) at via.xs​:46 #4 0x00716948 in PerlIOVia_method (my_perl=0x1811400\, f=0x1101954\,
method=0x71cf34 "GETARG"\, save=0x1137e58\, flags=0) at via.xs​:72 #5 0x0071ba6c in PerlIOVia_getarg (my_perl=0x1811400\, f=0x1101954\,
param=0xbfffebd8\, flags=1) at via.xs​:576 #6 0x00008ea4 in PerlIOBase_dup (my_perl=0x1811400\, f=0x1142a94\,
o=0x1101954\, param=0xbfffebd8\, flags=1) at perlio.c​:2250 #7 0x0071bac8 in PerlIOVia_dup (my_perl=0x1811400\, f=0x1142a94\,
o=0x1101954\, param=0xbfffebd8\, flags=1) at via.xs​:583 #8 0x000031ec in PerlIO_fdupopen (my_perl=0x1811400\, f=0x1101954\,
param=0xbfffebd8\, flags=1) at perlio.c​:561 #9 0x001b10f4 in Perl_fp_dup (my_perl=0x1811400\, fp=0x1101954\,
type=0 '\0'\, param=0xbfffebd8) at sv.c​:9499 #10 0x00003920 in PerlIO_clone (my_perl=0x1811400\, proto=0x1800400\,
param=0xbfffebd8) at perlio.c​:673 #11 0x001b9124 in perl_clone (proto_perl=0x1800400\, flags=2) at sv.c​: 10868 #12 0x005d70d0 in S_ithread_create (my_perl=0x1800400\,
init_function=0x18354b4\, stack_size=0\, gimme=0\, exit_opt=0\,
params=0x18357e8) at threads.xs​:649 #13 0x005da560 in XS_threads_create (my_perl=0x1800400\, cv=0x182ad60)
at threads.xs​:930 #14 0x00138fd4 in Perl_pp_goto (my_perl=0x1800400) at pp_ctl.c​:2415 #15 0x0027dad0 in Perl_runops_debug (my_perl=0x1800400) at dump.c​:1874 #16 0x00056db4 in S_run_body (my_perl=0x1800400\, oldscope=1) at
perl.c​:2403 #17 0x000560b4 in perl_run (my_perl=0x1800400) at perl.c​:2323 #18 0x00002a28 in main (argc=4\, argv=0xbffff804\, env=0xbffff818) at
perlmain.c​:113 ### end debugger log

Perl Info ``` Flags: category=library severity=high Site configuration information for perl 5.9.5: Configured by blgl at Sat Dec 9 12:49:18 CET 2006. Summary of my perl5 (revision 5 version 9 subversion 5 patch 29492) configuration: Platform: osname=darwin, osvers=8.8.0, archname=darwin-thread- multi-64int-2level uname='darwin sunwukung.local 8.8.0 darwin kernel version 8.8.0: fri sep 8 17:18:57 pdt 2006; root:xnu-792.12.6.obj~1release_ppc power macintosh powerpc ' config_args='' hint=previous, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=define, use64bitall=undef, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-fno-common -DPERL_DARWIN -no-cpp-precomp - DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include', optimize='-g', cppflags='-no-cpp-precomp -fno-common -DPERL_DARWIN -no-cpp- precomp -DDEBUGGING -fno-strict-aliasing -pipe -I/usr/local/include - fno-common -DPERL_DARWIN -no-cpp-precomp -DDEBUGGING -fno-strict- aliasing -pipe -I/usr/local/include' ccversion='', gccversion='4.0.1 (Apple Computer, Inc. build 5363)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=87654321 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags =' -L/usr/ local/lib' libpth=/usr/local/lib /usr/lib libs=-ldbm -ldl -lm -lc perllibs=-ldl -lm -lc libc=/usr/lib/libc.dylib, so=dylib, useshrplib=false, libperl=libperl.a gnulibc_version='' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup -L/ usr/local/lib' Locally applied patches: DEVEL @INC for perl 5.9.5: /usr/unstable/lib/perl5/5.9.5/darwin-thread-multi-64int-2level /usr/unstable/lib/perl5/5.9.5 /usr/unstable/lib/perl5/site_perl/5.9.5/darwin-thread- multi-64int-2level /usr/unstable/lib/perl5/site_perl/5.9.5 . Environment for perl 5.9.5: DYLD_LIBRARY_PATH (unset) HOME=/Users/blgl LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/Users/blgl/bin:/Users/blgl/bin/powerpc-apple-darwin:/usr/ local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/arla/bin PERL_BADLANG (unset) SHELL=/bin/tcsh ```
p5pRT commented 17 years ago

From blgl@hagernas.com

It turns out that PerlIO​::via and PerlIO​::encoding are both thread unsafe for exactly the same reason​: trying to execute Perl code in the new interpreter which is in the middle of being cloned. Attempted call graph​:

  PerlIOBase_dup   |   +---calls the Getarg virtual function   | |   | +---tries to execute Perl code   |   +---calls PerlIO_push   |   +---calls the Pushed virtual function   |   +---tries to execute Perl code

The least ugly solution I can think of involves adding a helper function which does everything PerlIOBase_dup does\, except for calling virtual functions. PerlIOVia_dup and PerlIOEncode_dup would (in the PERLIO_DUP_CLONE case) call this and then copy any extra data from the old instance to the new one with sv_dup as appropriate.

Any better ideas?

/Bo Lindbergh

p5pRT commented 8 years ago

From @dcollinsn

dcollins@​nightshade64​:\~/toolchain$ perl5.8.8-thread-multi 41106.pl Hello=3DWorld! Bus error dcollins@​nightshade64​:\~/toolchain$ perl5.25.2-thread-multi 41106.pl Hello=3DWorld! No package specified at 41106.pl line 12. No package specified at 41106.pl line 12. Filehandle $foo opened only for input at 41106.pl line 11. Unbalanced string table refcount​: (1) for "GETARG" during global destruction. Attempt to free nonexistent shared string 'GETARG'\, Perl interpreter​: 0x1f4a010 during global destruction.

-- Respectfully\, Dan Collins

p5pRT commented 7 years ago

From @jkeenan

On Mon\, 18 Dec 2006 17​:59​:12 GMT\, blgl@​hagernas.com wrote​:

This is a bug report for perl from blgl@​hagernas.com\, generated with the help of perlbug 1.35 running under perl 5.9.5.

----------------------------------------------------------------- [Please enter your report here]

This outputs "Hello=3DWorld!" as expected but then crashes.

### begin example program #! /usr/unstable/bin/perl

use strict; use warnings; use threads; use PerlIO​::via​::QuotedPrint;

open(my $foo\,'>&​:via(QuotedPrint)'\,*STDOUT) or die "open​: $!"; print $foo "Hello=World!\n" or die "print​: $!"; async { print $foo "Hello=from=Thread!\n"; }->join(); ### end example program

### begin debugger log Program received signal EXC_BAD_ACCESS\, Could not access memory. Reason​: KERN_INVALID_ADDRESS at address​: 0xabababaf 0x0008f580 in Perl_gv_fetchpvn_flags (my_perl=0x1811400\,
nambeg=0xbfffe520 "UNIVERSAL​::"\, full_len=11\, flags=0\, sv_type=12) at
gv.c​:815 815 if (!stash || !SvREFCNT(stash)) /* symbol table
under destruction */ (gdb) print stash $1 = (HV *) 0xabababab (gdb) bt #0 0x0008f580 in Perl_gv_fetchpvn_flags (my_perl=0x1811400\,
nambeg=0xbfffe520 "UNIVERSAL​::"\, full_len=11\, flags=0\, sv_type=12) at
gv.c​:815 #1 0x0008ece0 in Perl_gv_stashpvn (my_perl=0x1811400\, name=0x2d3bf0
"UNIVERSAL"\, namelen=11\, create=0) at gv.c​:736 #2 0x0008c410 in Perl_gv_fetchmeth (my_perl=0x1811400\,
stash=0x182af64\, name=0x71cf34 "GETARG"\, len=6\, level=0) at gv.c​:410 #3 0x007167b8 in PerlIOVia_fetchmethod (my_perl=0x1811400\,
s=0x1137e20\, method=0x71cf34 "GETARG"\, save=0x1137e58) at via.xs​:46 #4 0x00716948 in PerlIOVia_method (my_perl=0x1811400\, f=0x1101954\,
method=0x71cf34 "GETARG"\, save=0x1137e58\, flags=0) at via.xs​:72 #5 0x0071ba6c in PerlIOVia_getarg (my_perl=0x1811400\, f=0x1101954\,
param=0xbfffebd8\, flags=1) at via.xs​:576 #6 0x00008ea4 in PerlIOBase_dup (my_perl=0x1811400\, f=0x1142a94\,
o=0x1101954\, param=0xbfffebd8\, flags=1) at perlio.c​:2250 #7 0x0071bac8 in PerlIOVia_dup (my_perl=0x1811400\, f=0x1142a94\,
o=0x1101954\, param=0xbfffebd8\, flags=1) at via.xs​:583 #8 0x000031ec in PerlIO_fdupopen (my_perl=0x1811400\, f=0x1101954\,
param=0xbfffebd8\, flags=1) at perlio.c​:561 #9 0x001b10f4 in Perl_fp_dup (my_perl=0x1811400\, fp=0x1101954\,
type=0 '\0'\, param=0xbfffebd8) at sv.c​:9499 #10 0x00003920 in PerlIO_clone (my_perl=0x1811400\, proto=0x1800400\,
param=0xbfffebd8) at perlio.c​:673 #11 0x001b9124 in perl_clone (proto_perl=0x1800400\, flags=2) at sv.c​: 10868 #12 0x005d70d0 in S_ithread_create (my_perl=0x1800400\,
init_function=0x18354b4\, stack_size=0\, gimme=0\, exit_opt=0\,
params=0x18357e8) at threads.xs​:649 #13 0x005da560 in XS_threads_create (my_perl=0x1800400\, cv=0x182ad60)
at threads.xs​:930 #14 0x00138fd4 in Perl_pp_goto (my_perl=0x1800400) at pp_ctl.c​:2415 #15 0x0027dad0 in Perl_runops_debug (my_perl=0x1800400) at dump.c​:1874 #16 0x00056db4 in S_run_body (my_perl=0x1800400\, oldscope=1) at
perl.c​:2403 #17 0x000560b4 in perl_run (my_perl=0x1800400) at perl.c​:2323 #18 0x00002a28 in main (argc=4\, argv=0xbffff804\, env=0xbffff818) at
perlmain.c​:113 ### end debugger log

The way perl handles this appears to have been mitigated over the years since perl-5.9.5.

I added a one-line 'print' statement to your original program (see attachment).

I built threaded perls at several tags and ran your program at each.

##### [v5.12.5] $ ./bin/perl -Ilib ~/learn/perl/p5p/41106-threads-quoted-print.pl Hello=3DWorld! No package specified at /home/jkeenan/learn/perl/p5p/41106-threads-quoted-print.pl line 12. No package specified at /home/jkeenan/learn/perl/p5p/41106-threads-quoted-print.pl line 12. Filehandle $foo opened only for input at /home/jkeenan/learn/perl/p5p/41106-threads-quoted-print.pl line 11. Unbalanced string table refcount​: (1) for "/home/jkeenan/learn/perl/p5p/41106-threads-quoted-print.pl" during global destruction. Unbalanced string table refcount​: (1) for "_GEN_0" during global destruction. Unbalanced string table refcount​: (1) for "GETARG" during global destruction. Scalars leaked​: 1 Finished Segmentation fault (core dumped)

[v5.16.3] $ ./bin/perl -Ilib ~/learn/perl/p5p/41106-threads-quoted-print.pl Hello=3DWorld! No package specified at /home/jkeenan/learn/perl/p5p/41106-threads-quoted-print.pl line 12. No package specified at /home/jkeenan/learn/perl/p5p/41106-threads-quoted-print.pl line 12. Filehandle $foo opened only for input at /home/jkeenan/learn/perl/p5p/41106-threads-quoted-print.pl line 11. Unbalanced string table refcount​: (1) for "GETARG" during global destruction. Finished Attempt to free nonexistent shared string '0'\, Perl interpreter​: 0x260b010 during global destruction.

[v5.20.3] $ ./bin/perl -Ilib ~/learn/perl/p5p/41106-threads-quoted-print.pl Hello=3DWorld! No package specified at /home/jkeenan/learn/perl/p5p/41106-threads-quoted-print.pl line 12. No package specified at /home/jkeenan/learn/perl/p5p/41106-threads-quoted-print.pl line 12. Filehandle $foo opened only for input at /home/jkeenan/learn/perl/p5p/41106-threads-quoted-print.pl line 11. Unbalanced string table refcount​: (1) for "GETARG" during global destruction. Finished

[blead​: commit 8df0224] same results as v5.20.3 #####

So the segfault disappeared somewhere between 5.12.5 and 5.16.3. From some point between 5.16.3 and 5.20.3\, we start to get fairly sane warnings.

Is there any reason to keep this ticket open?

Thank you very much.

-- James E Keenan (jkeenan@​cpan.org)

p5pRT commented 7 years ago

From @jkeenan

41106-threads-quoted-print.pl

p5pRT commented 7 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 7 years ago

From @iabyn

On Sat\, Dec 31\, 2016 at 06​:56​:05AM -0800\, James E Keenan via RT wrote​:

[v5.20.3] $ ./bin/perl -Ilib ~/learn/perl/p5p/41106-threads-quoted-print.pl .... Unbalanced string table refcount​: (1) for "GETARG" during global destruction.

[blead​: commit 8df0224] same results as v5.20.3

So the segfault disappeared somewhere between 5.12.5 and 5.16.3. From some point between 5.16.3 and 5.20.3\, we start to get fairly sane warnings.

The 'Unbalanced string table refcount' error indicates that there is still at least one bug.

-- Little fly\, thy summer's play my thoughtless hand has terminated with extreme prejudice.   (with apologies to William Blake)