search

Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.98k stars 559 forks source link

use encoding "utf8" causes segfault #7533

Closed p5pRT closed 9 years ago

p5pRT commented 20 years ago

Migrated from rt.perl.org#31923 (status was 'resolved')

Searchable as RT31923$

p5pRT commented 20 years ago

From hatta@yandex.ru

Hello\, Today I was playing with gtk2-perl and found that for some reason my program\, running two threads - gui and worker - crashed when thread->new was executed. So i began investigating this and found\, that gtk2-perl was not the cause of segfault\, i found that it was all use encoding "utf8" statement​:

this code gives segmentation fault on my Fedora Core 2/linux-2.6.8.1

#!/usr/bin/perl -w use strict; use warnings; use threads; use threads​::shared; use encoding "utf8"; # no segfault if remove

my $t = threads->new(\&fn);

sub fn {   print "hello\n"; }

Output from perlbug -d (I cannot send email from my local machine)

Flags​:   category=   severity=

Site configuration information for perl v5.8.3​:

Configured by bhcompile at Thu Apr 15 13​:08​:28 EDT 2004.

Summary of my perl5 (revision 5.0 version 8 subversion 3) configuration​:   Platform​:   osname=linux\, osvers=2.4.21-4.elsmp\, archname=i386-linux-thread-multi   uname='linux tweety.devel.redhat.com 2.4.21-4.elsmp #1 smp fri oct 3 17​:52​:56 edt 2003 i686 i686 i386 gnulinux '   config_args='-des -Doptimize=-O2 -g -pipe -march=i386 -mcpu=i686 -Dversion=5.8.3 -Dmyhostname=localhost -Dperladmin=root@​localhost -Dcc=gcc -Dcf_by=Red Hat\, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux -Dvendorprefix=/usr -Dsiteprefix=/usr -Duseshrplib -Dusethreads -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dinc_version_list=5.8.2 5.8.1 5.8.0'   hint=recommended\, useposix=true\, d_sigaction=define   usethreads=define use5005threads=undef useithreads=define usemultiplicity=define   useperlio=define d_sfio=undef uselargefiles=define usesocks=undef   use64bitint=undef use64bitall=undef uselongdouble=undef   usemymalloc=n\, bincompat5005=undef   Compiler​:   cc='gcc'\, ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm'\,   optimize='-O2 -g -pipe -march=i386 -mcpu=i686'\,   cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm'   ccversion=''\, gccversion='3.3.3 20040412 (Red Hat Linux 3.3.3-7)'\, gccosandvers=''   intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234   d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=12   ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8   alignbytes=4\, prototype=define   Linker and Libraries​:   ld='gcc'\, ldflags =' -L/usr/local/lib'   libpth=/usr/local/lib /lib /usr/lib   libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc   perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc   libc=/lib/libc-2.3.3.so\, so=so\, useshrplib=true\, libperl=libperl.so   gnulibc_version='2.3.3'   Dynamic Linking​:   dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-rdynamic -Wl\,-rpath\,/usr/lib/perl5/5.8.3/i386-linux-thread-multi/CORE'   cccdlflags='-fPIC'\, lddlflags='-shared -L/usr/local/lib'

Locally applied patches​:

@​INC for perl v5.8.3​:   /usr/lib/perl5/5.8.3/i386-linux-thread-multi   /usr/lib/perl5/5.8.3   /usr/lib/perl5/site_perl/5.8.3/i386-linux-thread-multi   /usr/lib/perl5/site_perl/5.8.2/i386-linux-thread-multi   /usr/lib/perl5/site_perl/5.8.1/i386-linux-thread-multi   /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi   /usr/lib/perl5/site_perl/5.8.3   /usr/lib/perl5/site_perl/5.8.2   /usr/lib/perl5/site_perl/5.8.1   /usr/lib/perl5/site_perl/5.8.0   /usr/lib/perl5/site_perl   /usr/lib/perl5/vendor_perl/5.8.3/i386-linux-thread-multi   /usr/lib/perl5/vendor_perl/5.8.2/i386-linux-thread-multi   /usr/lib/perl5/vendor_perl/5.8.1/i386-linux-thread-multi   /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi   /usr/lib/perl5/vendor_perl/5.8.3   /usr/lib/perl5/vendor_perl/5.8.2   /usr/lib/perl5/vendor_perl/5.8.1   /usr/lib/perl5/vendor_perl/5.8.0   /usr/lib/perl5/vendor_perl   .

Environment for perl v5.8.3​:   HOME=/home/hatta   LANG=ru_RU.UTF-8   LANGUAGE (unset)   LD_LIBRARY_PATH (unset)   LOGDIR (unset)

PATH=/usr/kerberos/bin​:/usr/local/bin​:/usr/bin​:/bin​:/usr/X11R6/bin​:/usr/lib/java/jre/bin​:/usr/lib/java/bin​:/usr/local/Wine/bin   PERL_BADLANG (unset)   SHELL=/bin/bash

Goodbye.

-- K. S. \mailto​:hatta@​NOT\-FOR\-SPAM\.yandex\.ru ICQ​: 224295216 Jabber​: hatta@​jabber.ru

p5pRT commented 20 years ago

From @rgs

Konstantin Stopani (via RT) wrote​:

this code gives segmentation fault on my Fedora Core 2/linux-2.6.8.1

#!/usr/bin/perl -w use strict; use warnings; use threads; use threads​::shared; use encoding "utf8"; # no segfault if remove

my $t = threads->new(\&fn);

sub fn { print "hello\n"; }

Reproduced here with bleadperl; apparently this comes from this bit in ext/PerlIO/encoding/encoding.xs (function PerlIOEncode_getarg()) :

  /* Not 100% sure stack swap is right thing to do during dup ... */   PUSHSTACKi(PERLSI_MAGIC);

p5pRT commented 20 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 20 years ago

From nick@ing-simmons.net

Rafael Garcia-Suarez \rgarciasuarez@​mandrakesoft\.com writes​:

Konstantin Stopani (via RT) wrote​:

this code gives segmentation fault on my Fedora Core 2/linux-2.6.8.1

#!/usr/bin/perl -w use strict; use warnings; use threads; use threads​::shared; use encoding "utf8"; # no segfault if remove

my $t = threads->new(\&fn);

sub fn { print "hello\n"; }

Reproduced here with bleadperl; apparently this comes from this bit in ext/PerlIO/encoding/encoding.xs (function PerlIOEncode_getarg()) :

   /\* Not 100% sure stack swap is right thing to do during dup \.\.\. \*/

Seems like it isn't ;-) 

   PUSHSTACKi\(PERLSI\_MAGIC\);

Someone could experiment with commenting out the stack swap\, that isn't the right total fix but if it made segfault go away it would be a hint.

p5pRT commented 19 years ago

From @kimurakoichi

Hi\, I use perl with sjis encoding. Segfault occures when use encoding with thread.

This cause by the script above.

use v5.8.0; use strict; use threads; use encoding 'ascii';

my $thread1 = threads->new(\&main\, "One"); my $thread2 = threads->new(\&main\, "Two");

$thread1->join; $thread2->join;

print "done.\n";

sub main {   my ($name) = @​_;   for (1..10) {   print "I am $name ($_)\n";   threads->yield();   } }

Summary of my perl5 (revision 5 version 8 subversion 6) configuration​:   Platform​:   osname=cygwin\, osvers=1.5.12(0.11642)\, archname=cygwin-thread-multi-64int   uname='cygwin_nt-4.0 loreley 1.5.12(0.11642) 2004-11-10 08​:34 i686 unknown unknown cygwin '   config_args='-de -Dmksymlinks -Duse64bitint -Dusethreads -Doptimize=-O3 -Dman3ext=3pm'   hint=recommended\, useposix=true\, d_sigaction=define   usethreads=define use5005threads=undef useithreads=define usemultiplicity=define   useperlio=define d_sfio=undef uselargefiles=define usesocks=undef   use64bitint=define use64bitall=undef uselongdouble=undef   usemymalloc=y\, bincompat5005=undef   Compiler​:   cc='gcc'\, ccflags ='-DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -I/usr/local/include'\,   optimize='-O3'\,   cppflags='-DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -I/usr/local/include'   ccversion=''\, gccversion='3.4.1 (cygming special)'\, gccosandvers=''   intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=12345678   d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=12   ivtype='long long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8   alignbytes=8\, prototype=define   Linker and Libraries​:   ld='ld2'\, ldflags =' -s -L/usr/local/lib'   libpth=/usr/local/lib /usr/lib /lib   libs=-lgdbm -ldb -lcrypt -lgdbm_compat   perllibs=-lcrypt -lgdbm_compat   libc=/usr/lib/libc.a\, so=dll\, useshrplib=true\, libperl=libperl.a   gnulibc_version=''   Dynamic Linking​:   dlsrc=dl_dlopen.xs\, dlext=dll\, d_dlsymun=undef\, ccdlflags=' -s'   cccdlflags=' '\, lddlflags=' -s -L/usr/local/lib'

Characteristics of this binary (from libperl)​:   Compile-time options​: MULTIPLICITY USE_ITHREADS USE_64_BIT_INT USE_LARGE_FILES PERL_IMPLICIT_CONTEXT   Built under cygwin   Compiled at Jan 12 2005 01​:07​:08   %ENV​:   PERL5LIB="d​:/home/perl"   CYGWIN=""   @​INC​:   d   /home/perl   /usr/lib/perl5/5.8/cygwin   /usr/lib/perl5/5.8   /usr/lib/perl5/site_perl/5.8/cygwin   /usr/lib/perl5/site_perl/5.8   /usr/lib/perl5/site_perl/5.8   /usr/lib/perl5/vendor_perl/5.8/cygwin   /usr/lib/perl5/vendor_perl/5.8   /usr/lib/perl5/vendor_perl/5.8   .

thanks\,

-- Koichi Kimura   I thought what I'd do was\, I'd pretend I was one of those deaf-mutes.   mail kbk@​kt.rim.or.jp   web www.kt.rim.or.jp/~kbk/index.html#zakkicho   homepage3.nifty.com/farstar/

p5pRT commented 17 years ago

From rkitover@walmart.com

Created by rkitover@rkitover-linux.walmart.com

As reported by DawnFantasy on IRC (FreeNode #perl)​: /usr/local/bin/perl -Mthreads -e 'binmode STDOUT\, "​:encoding(UTF-8)"; threads->new( sub {sleep 3;print "K\n";}); sleep 5'

produces a core dump.

5.8.8 (Debian unstable version) produces a core dump as well.

Perl Info ``` Flags: category=library severity=medium Site configuration information for perl 5.9.5: Configured by rkitover at Thu Aug 23 02:54:48 PDT 2007. Summary of my perl5 (revision 5 version 9 subversion 5 patch 31749) configuration: Platform: osname=linux, osvers=2.6.18-4-amd64, archname=x86_64-linux-thread-multi uname='linux rkitover-linux 2.6.18-4-amd64 #1 smp mon mar 26 11:36:53 cest 2007 x86_64 gnulinux ' config_args='-Dusedevel -Dprefix=/usr/local/stow/bleadperl -Dmad=y -Dusethreads -des' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2', cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -I/usr/local/include' ccversion='', gccversion='4.1.3 20070718 (prerelease) (Debian 4.1.2-14)', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib /lib64 /usr/lib64 /usr/local/lib64 libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc libc=/lib/libc-2.6.1.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.6.1' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib' Locally applied patches: DEVEL @INC for perl 5.9.5: /usr/local/stow/bleadperl/lib/5.9.5/x86_64-linux-thread-multi /usr/local/stow/bleadperl/lib/5.9.5 /usr/local/stow/bleadperl/lib/site_perl/5.9.5/x86_64-linux-thread-multi /usr/local/stow/bleadperl/lib/site_perl/5.9.5 . Environment for perl 5.9.5: HOME=/home/rkitover LANG=en_US.UTF-8 LANGUAGE (unset) LC_CTYPE=en_US.utf8 LD_LIBRARY_PATH=/lib:/usr/lib:/usr/local/lib:/home/oracle/instantclient_10_2 LOGDIR (unset) PATH=/home/rkitover/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/bin/X11:/usr/games:/home/oracle/instantclient_10_2 PERL_BADLANG (unset) SHELL=zsh ```
p5pRT commented 17 years ago

From @jdhedden

As reported by DawnFantasy on IRC (FreeNode #perl)​: /usr/local/bin/perl -Mthreads -e 'binmode STDOUT\, "​:encoding(UTF-8)"; threads->new( sub {sleep 3;print "K\n";}); sleep 5'

produces a core dump.

5.8.8 (Debian unstable version) produces a core dump as well.

Produces a seg fault with blead under Cygwin on Windows.

p5pRT commented 17 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 17 years ago

From shouldbedomo@mac.com

On 2007–08–23\, at 13​:52\, Jerry D. Hedden wrote​:

As reported by DawnFantasy on IRC (FreeNode #perl)​: /usr/local/bin/perl -Mthreads -e 'binmode STDOUT\, "​:encoding (UTF-8)"; threads->new( sub {sleep 3;print "K\n";}); sleep 5'

produces a core dump.

5.8.8 (Debian unstable version) produces a core dump as well.

Produces a seg fault with blead under Cygwin on Windows.

And on Mac OS 10.4.10 with both 5.8.8 and bleadperl (5.10 to be).
Here's the log from 5.8.8 (that from blead is very similar indeed)​:

Host Name​: Tullamore Date/Time​: 2007-08-23 14​:22​:22.390 +0200 OS Version​: 10.4.10 (Build 8R218) Report Version​: 4

Command​: perl Path​: /usr/local/bin/perl Parent​: bash [8591]

Version​: ??? (???)

PID​: 11777 Thread​: 0

Exception​: EXC_BAD_ACCESS (0x0001) Codes​: KERN_PROTECTION_FAILURE (0x0002) at 0x00000018

Thread 0 Crashed​: 0 encoding.bundle 0x000262ac PerlIOEncode_getarg + 76 1 libperl.dylib 0x002ee548 PerlIOBase_dup + 248 2 encoding.bundle 0x00028c10 PerlIOEncode_dup + 32 3 libperl.dylib 0x0027e80c Perl_fp_dup + 156 4 libperl.dylib 0x002f0f30 PerlIO_clone + 560 5 libperl.dylib 0x002874d8 perl_clone + 3352 6 threads.bundle 0x0001efd4 XS_threads_create + 4852 7 libperl.dylib 0x0027cdb4 Perl_pp_entersub + 1764 8 libperl.dylib 0x00273f08 Perl_runops_standard + 88 9 libperl.dylib 0x0020926c perl_run + 524 10 perl 0x00002910 main + 240 11 perl 0x0000243c _start + 760 12 perl 0x00002140 start + 48

Thread 0 crashed with PPC Thread State 64​:   srr0​: 0x00000000000262ac srr1​:
0x100000000200f030 vrsave​: 0x0000000000000000   cr​: 0x22008274 xer​: 0x0000000000000000 lr​:
0x00000000000262a8 ctr​: 0x00000000002f7030   r0​: 0x00000000000262a8 r1​: 0x00000000bffff0e0 r2​:
0x0000000000000000 r3​: 0x0000000001841b88   r4​: 0x0000000000401ac8 r5​: 0x00000000bffff318 r6​:
0x0000000000000001 r7​: 0x00000000bffff318   r8​: 0x0000000000000000 r9​: 0x00000000003212e4 r10​:
0x0000000000000600 r11​: 0x000000000002a0b8   r12​: 0x00000000002f7030 r13​: 0x000000000043fae0 r14​:
0x000000000182a4a8 r15​: 0x0000000000000000   r16​: 0x000000000043fa80 r17​: 0x0000000000000002 r18​:
0x0000000001842344 r19​: 0x0000000001842350   r20​: 0x000000000184235c r21​: 0x0000000000000001 r22​:
0x0000000001800400 r23​: 0x0000000001800400   r24​: 0x0000000000000001 r25​: 0x0000000001842344 r26​:
0x0000000000440e80 r27​: 0x0000000000000000   r28​: 0x0000000001841a00 r29​: 0x000000000002a004 r30​:
0x0000000000441c68 r31​: 0x0000000000026270

Binary Images Description​:   0x1000 - 0x3fff perl /usr/local/bin/perl   0x1a000 - 0x21fff threads.bundle /usr/local/lib/perl5/5.8.8/ darwin-thread-multi-2level/auto/threads/threads.bundle   0x25000 - 0x29fff encoding.bundle /usr/local/lib/perl5/5.8.8/ darwin-thread-multi-2level/auto/PerlIO/encoding/encoding.bundle   0x2d000 - 0x33fff Encode.bundle /usr/local/lib/perl5/5.8.8/ darwin-thread-multi-2level/auto/Encode/Encode.bundle   0x205000 - 0x317fff libperl.dylib /usr/local/lib/perl5/5.8.8/ darwin-thread-multi-2level/CORE/libperl.dylib 0x8fe00000 - 0x8fe52fff dyld 46.12 /usr/lib/dyld 0x90000000 - 0x901bcfff libSystem.B.dylib /usr/lib/libSystem.B.dylib 0x94c90000 - 0x94cb0fff libmx.A.dylib /usr/lib/libmx.A.dylib 0x9611d000 - 0x96122fff libmathCommon.A.dylib /usr/lib/system/ libmathCommon.A.dylib

-- Dominic Dunlop

p5pRT commented 17 years ago

From @smpeters

On Thu\, Aug 23\, 2007 at 03​:45​:26AM -0700\, Rafael Kitover wrote​:

# New Ticket Created by "Rafael Kitover" # Please include the string​: [perl #44887] # in the subject line of all future correspondence about this issue. # \<URL​: http​://rt.perl.org/rt3/Ticket/Display.html?id=44887 >

This is a bug report for perl from rkitover@​rkitover-linux.walmart.com\, generated with the help of perlbug 1.36 running under perl 5.9.5.

----------------------------------------------------------------- [Please enter your report here]

As reported by DawnFantasy on IRC (FreeNode #perl)​: /usr/local/bin/perl -Mthreads -e 'binmode STDOUT\, "​:encoding(UTF-8)"; threads->new( sub {sleep 3;print "K\n";}); sleep 5'

produces a core dump.

5.8.8 (Debian unstable version) produces a core dump as well.

Here's the backtrace with blead....

Program terminated with signal 11\, Segmentation fault. #0 0x00c0afa1 in PerlIOEncode_getarg (my_perl=0x8e44108\, f=0x8dfffec\,   param=0xbfb7d6e4\, flags=1) at encoding.xs​:64 64 PUSHSTACKi(PERLSI_MAGIC); (gdb) bt #0 0x00c0afa1 in PerlIOEncode_getarg (my_perl=0x8e44108\, f=0x8dfffec\,   param=0xbfb7d6e4\, flags=1) at encoding.xs​:64 #1 0x0822f988 in PerlIOBase_dup (my_perl=0x8e44108\, f=0x8e685b4\, o=0x8dfffec\,   param=0xbfb7d6e4\, flags=1) at perlio.c​:2269 #2 0x00c125af in PerlIOEncode_dup (my_perl=0x8e44108\, f=0x8e685b4\,   o=0x8dfffec\, params=0xbfb7d6e4\, flags=1) at encoding.xs​:554 #3 0x0822bedf in PerlIO_fdupopen (my_perl=0x8e44108\, f=0x8dfffec\,   param=0xbfb7d6e4\, flags=1) at perlio.c​:569 #4 0x08153680 in Perl_fp_dup (my_perl=0x8e44108\, fp=0x8dfffec\, type=0 '\0'\,   param=0xbfb7d6e4) at sv.c​:9669 #5 0x0822c32a in PerlIO_clone (my_perl=0x8e44108\, proto=0x8de7008\,   param=0xbfb7d6e4) at perlio.c​:685 #6 0x0815a576 in perl_clone (proto_perl=0x8de7008\, flags=2) at sv.c​:11049 #7 0x004e7588 in S_ithread_create (my_perl=0x8de7008\,   init_function=0x8deb7b4\, stack_size=0\, gimme=128\, exit_opt=0\,   params=0x8e463a4) at threads.xs​:666 #8 0x004eade4 in XS_threads_create (my_perl=0x8de7008\, cv=0x8e45ca4)   at threads.xs​:953 #9 0x0811d764 in Perl_pp_entersub (my_perl=0x8de7008) at pp_hot.c​:2823 #10 0x080c3e78 in Perl_runops_debug (my_perl=0x8de7008) at dump.c​:1918 #11 0x080fe0e8 in S_run_body (my_perl=0x8de7008\, oldscope=1) at perl.c​:2429 #12 0x080fd54d in perl_run (my_perl=0x8de7008) at perl.c​:2347 #13 0x0806068c in main (argc=5\, argv=0xbfb7dd44\, env=0xbfb7dd5c)

p5pRT commented 17 years ago

From @smpeters

On Thu\, Aug 23\, 2007 at 08​:01​:56AM -0500\, Steve Peters wrote​:

On Thu\, Aug 23\, 2007 at 03​:45​:26AM -0700\, Rafael Kitover wrote​:

# New Ticket Created by "Rafael Kitover" # Please include the string​: [perl #44887] # in the subject line of all future correspondence about this issue. # \<URL​: http​://rt.perl.org/rt3/Ticket/Display.html?id=44887 >

This is a bug report for perl from rkitover@​rkitover-linux.walmart.com\, generated with the help of perlbug 1.36 running under perl 5.9.5.

----------------------------------------------------------------- [Please enter your report here]

As reported by DawnFantasy on IRC (FreeNode #perl)​: /usr/local/bin/perl -Mthreads -e 'binmode STDOUT\, "​:encoding(UTF-8)"; threads->new( sub {sleep 3;print "K\n";}); sleep 5'

produces a core dump.

5.8.8 (Debian unstable version) produces a core dump as well.

Here's the backtrace with blead....

Program terminated with signal 11\, Segmentation fault. #0 0x00c0afa1 in PerlIOEncode_getarg (my_perl=0x8e44108\, f=0x8dfffec\, param=0xbfb7d6e4\, flags=1) at encoding.xs​:64 64 PUSHSTACKi(PERLSI_MAGIC); (gdb) bt #0 0x00c0afa1 in PerlIOEncode_getarg (my_perl=0x8e44108\, f=0x8dfffec\, param=0xbfb7d6e4\, flags=1) at encoding.xs​:64 #1 0x0822f988 in PerlIOBase_dup (my_perl=0x8e44108\, f=0x8e685b4\, o=0x8dfffec\, param=0xbfb7d6e4\, flags=1) at perlio.c​:2269 #2 0x00c125af in PerlIOEncode_dup (my_perl=0x8e44108\, f=0x8e685b4\, o=0x8dfffec\, params=0xbfb7d6e4\, flags=1) at encoding.xs​:554 #3 0x0822bedf in PerlIO_fdupopen (my_perl=0x8e44108\, f=0x8dfffec\, param=0xbfb7d6e4\, flags=1) at perlio.c​:569 #4 0x08153680 in Perl_fp_dup (my_perl=0x8e44108\, fp=0x8dfffec\, type=0 '\0'\, param=0xbfb7d6e4) at sv.c​:9669 #5 0x0822c32a in PerlIO_clone (my_perl=0x8e44108\, proto=0x8de7008\, param=0xbfb7d6e4) at perlio.c​:685 #6 0x0815a576 in perl_clone (proto_perl=0x8de7008\, flags=2) at sv.c​:11049 #7 0x004e7588 in S_ithread_create (my_perl=0x8de7008\, init_function=0x8deb7b4\, stack_size=0\, gimme=128\, exit_opt=0\, params=0x8e463a4) at threads.xs​:666 #8 0x004eade4 in XS_threads_create (my_perl=0x8de7008\, cv=0x8e45ca4) at threads.xs​:953 #9 0x0811d764 in Perl_pp_entersub (my_perl=0x8de7008) at pp_hot.c​:2823 #10 0x080c3e78 in Perl_runops_debug (my_perl=0x8de7008) at dump.c​:1918 #11 0x080fe0e8 in S_run_body (my_perl=0x8de7008\, oldscope=1) at perl.c​:2429 #12 0x080fd54d in perl_run (my_perl=0x8de7008) at perl.c​:2347 #13 0x0806068c in main (argc=5\, argv=0xbfb7dd44\, env=0xbfb7dd5c)

Now that I look at the code\, this is a known bug. I'll merge this in with RT #31923.

Steve Peters steve@​fisharerojo.org

p5pRT commented 17 years ago

From blgl@hagernas.com

In article \20070823130816\.GA3683@&#8203;kirk\.peters\.homeunix\.org\, steve@​fisharerojo.org (Steve Peters) wrote​:

Now that I look at the code\, this is a known bug. I'll merge this in with RT #31923.

It's related to #41106 as well.

/Bo Lindbergh

p5pRT commented 13 years ago

From @nwc10

Created by @nwc10

$ valgrind ./perl -Ilib -Mthreads -e '{use encoding "latin1"} threads->new(sub {})->join()' ==17052== Memcheck\, a memory error detector. ==17052== Copyright (C) 2002-2007\, and GNU GPL'd\, by Julian Seward et al. ==17052== Using LibVEX rev 1854\, a library for dynamic binary translation. ==17052== Copyright (C) 2004-2007\, and GNU GPL'd\, by OpenWorks LLP. ==17052== Using valgrind-3.3.1-Debian\, a dynamic binary instrumentation framework. ==17052== Copyright (C) 2000-2007\, and GNU GPL'd\, by Julian Seward et al. ==17052== For more details\, rerun with​: -v ==17052== ==17052== Invalid read of size 8 ==17052== at 0x668308A​: PerlIOEncode_getarg (in /home/nick/Perl/perl/lib/auto/PerlIO/encoding/encoding.so) ==17052== by 0x4E7767​: PerlIOBase_dup (in /home/nick/Perl/perl/perl) ==17052== by 0x66815EF​: PerlIOEncode_dup (in /home/nick/Perl/perl/lib/auto/PerlIO/encoding/encoding.so) ==17052== by 0x48D03C​: Perl_fp_dup (in /home/nick/Perl/perl/perl) ==17052== by 0x4E8CBC​: PerlIO_clone (in /home/nick/Perl/perl/perl) ==17052== by 0x49A892​: perl_clone (in /home/nick/Perl/perl/perl) ==17052== by 0x6274C33​: XS_threads_create (in /home/nick/Perl/perl/lib/auto/threads/threads.so) ==17052== by 0x485B8E​: Perl_pp_entersub (in /home/nick/Perl/perl/perl) ==17052== by 0x484419​: Perl_runops_standard (in /home/nick/Perl/perl/perl) ==17052== by 0x42F534​: perl_run (in /home/nick/Perl/perl/perl) ==17052== by 0x41C978​: main (in /home/nick/Perl/perl/perl) ==17052== Address 0x18 is not stack'd\, malloc'd or (recently) free'd ==17052== ==17052== Process terminating with default action of signal 11 (SIGSEGV) ==17052== Access not within mapped region at address 0x18 ==17052== at 0x668308A​: PerlIOEncode_getarg (in /home/nick/Perl/perl/lib/auto/PerlIO/encoding/encoding.so) ==17052== by 0x4E7767​: PerlIOBase_dup (in /home/nick/Perl/perl/perl) ==17052== by 0x66815EF​: PerlIOEncode_dup (in /home/nick/Perl/perl/lib/auto/PerlIO/encoding/encoding.so) ==17052== by 0x48D03C​: Perl_fp_dup (in /home/nick/Perl/perl/perl) ==17052== by 0x4E8CBC​: PerlIO_clone (in /home/nick/Perl/perl/perl) ==17052== by 0x49A892​: perl_clone (in /home/nick/Perl/perl/perl) ==17052== by 0x6274C33​: XS_threads_create (in /home/nick/Perl/perl/lib/auto/threads/threads.so) ==17052== by 0x485B8E​: Perl_pp_entersub (in /home/nick/Perl/perl/perl) ==17052== by 0x484419​: Perl_runops_standard (in /home/nick/Perl/perl/perl) ==17052== by 0x42F534​: perl_run (in /home/nick/Perl/perl/perl) ==17052== by 0x41C978​: main (in /home/nick/Perl/perl/perl) ==17052== ==17052== ERROR SUMMARY​: 1 errors from 1 contexts (suppressed​: 38 from 2) ==17052== malloc/free​: in use at exit​: 1\,206\,133 bytes in 14\,483 blocks. ==17052== malloc/free​: 27\,110 allocs\, 12\,627 frees\, 2\,160\,854 bytes allocated. ==17052== For counts of detected errors\, rerun with​: -v ==17052== searching for pointers to 14\,483 not-freed blocks. ==17052== checked 1\,407\,560 bytes. ==17052== ==17052== LEAK SUMMARY​: ==17052== definitely lost​: 2\,420 bytes in 36 blocks. ==17052== possibly lost​: 0 bytes in 0 blocks. ==17052== still reachable​: 1\,203\,713 bytes in 14\,447 blocks. ==17052== suppressed​: 0 bytes in 0 blocks. ==17052== Rerun with --leak-check=full to see details of leaked memory. Segmentation fault

Obviously it shouldn't do that.

Nicholas Clark

Perl Info ``` Flags: category=core severity=low Site configuration information for perl 5.13.10: Configured by nick at Sun Mar 20 10:09:17 GMT 2011. Summary of my perl5 (revision 5 version 13 subversion 10) configuration: Commit id: 75da9d4c616bae3e6791af93d2ced52dc8080f06 Platform: osname=linux, osvers=2.6.35.4, archname=x86_64-linux-thread-multi uname='linux eris 2.6.35.4 #4 smp tue sep 21 09:54:22 bst 2010 x86_64 gnulinux ' config_args='-Dusedevel=y -Dcc=ccache gcc -Dld=gcc -Ubincompat5005 -Uinstallusrbinperl -Dcf_email=nick@ccl4.org -Dperladmin=nick@ccl4.org -Dinc_version_list= -Dinc_version_list_init=0 -Doptimize=-Os -Dusethreads -Uuselongdouble -Uuse64bitall -Uusemymalloc -Duseperlio -Dprefix=~/Sandpit/snap5.9.x-v5.13.10-526-g88ecb8a -Uusevendorprefix -Uvendorprefix=~/Sandpit/snap5.9.x-v5.13.10-526-g88ecb8a -Dinstallman1dir=none -Dinstallman3dir=none -Uuserelocatableinc -Umad -de' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=define, use64bitall=undef, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='ccache gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-Os', cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include' ccversion='', gccversion='4.3.2', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='gcc', ldflags =' -fstack-protector -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib /lib64 /usr/lib64 libs=-lnsl -ldb -ldl -lm -lcrypt -lutil -lpthread -lc perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc libc=/lib/libc-2.7.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.7' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -Os -L/usr/local/lib -fstack-protector' Locally applied patches: @INC for perl 5.13.10: lib /home/nick/Sandpit/snap5.9.x-v5.13.10-526-g88ecb8a/lib/perl5/site_perl/5.13.10/x86_64-linux-thread-multi /home/nick/Sandpit/snap5.9.x-v5.13.10-526-g88ecb8a/lib/perl5/site_perl/5.13.10 /home/nick/Sandpit/snap5.9.x-v5.13.10-526-g88ecb8a/lib/perl5/5.13.10/x86_64-linux-thread-multi /home/nick/Sandpit/snap5.9.x-v5.13.10-526-g88ecb8a/lib/perl5/5.13.10 . Environment for perl 5.13.10: HOME=/home/nick LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/nick/bin:/usr/local/bin:/usr/bin:/bin:/usr/games:/usr/local/sbin:/sbin:/usr/sbin PERL_BADLANG (unset) SHELL=/bin/bash ```
p5pRT commented 13 years ago

From @nwc10

On Sun Mar 20 04​:04​:25 2011\, nicholas wrote​:

$ valgrind ./perl -Ilib -Mthreads -e '{use encoding "latin1"} threads-

new(sub {})->join()' Obviously it shouldn't do that.

The crash happens when trying to swap stacks here​:

  if (e->enc) {   dSP;   /* Not 100% sure stack swap is right thing to do during dup ... */   PUSHSTACKi(PERLSI_MAGIC);

[that code is called both during dup\, and at normal times]

That line was added in 2002 in 24f59afc531955e5c3768c6023fb2ac247d6939d The testcase crashes at that revision. It crashes at the next statement (SPAGAIN;) in the parent revision. This bug is old. The fix isn't going to be as simple as removing the stack swapping code.

Nicholas Clark

p5pRT commented 13 years ago

@nwc10 - Status changed from 'new' to 'open'

p5pRT commented 13 years ago

From blgl@stacken.kth.se

Yet another duplicate of #31923. PerlIO​::encoding isn't thread safe.

/Bo Lindbergh

p5pRT commented 12 years ago

From au@hcsd.de

Hello\,

this bug is still present in 5.14.2 - I've just hit it. Are there any plans to fix it?

Best regards\,

Stephan

p5pRT commented 12 years ago

From [Unknown Contact. See original ticket]

Hello\,

this bug is still present in 5.14.2 - I've just hit it. Are there any plans to fix it?

Best regards\,

Stephan

p5pRT commented 12 years ago

From au@hcsd.de

Hi\,

since I am facing this bug again within just a few weeks I took a closer look at encoding.xs and PerlIOEncode_getarg(). Unfortunately I don't understand much here\, the macros are not really well documented and it looks like I'm missing some background knowledge.

I'd be very happy if someone with a more in-depth knowledge could take some time for looking at the code and this long standing bug (reported in 2004). Sometimes threads and PerlIO​::encoding are just useful...

Thank you\, best regards\,

Stephan

On Fri Dec 09 05​:17​:52 2011\, stephan_a wrote​:

Hello\,

this bug is still present in 5.14.2 - I've just hit it. Are there any plans to fix it?

Best regards\,

Stephan

p5pRT commented 12 years ago

From @Leont

On Wed Dec 21 05​:52​:03 2011\, stephan_a wrote​:

since I am facing this bug again within just a few weeks I took a closer look at encoding.xs and PerlIOEncode_getarg(). Unfortunately I don't understand much here\, the macros are not really well documented and it looks like I'm missing some background knowledge.

I'd be very happy if someone with a more in-depth knowledge could take some time for looking at the code and this long standing bug (reported in 2004). Sometimes threads and PerlIO​::encoding are just useful...

I seems calling methods during thread duplications is a bad idea\, so the easy solution would be to just not do that. Instead\, the argument should be saved in the PerlIOEncode struct\, and PerlIO_getargs should just return a duplicate of it.

Leon

p5pRT commented 12 years ago

From pm.20.browseruk@xoxy.net

Created by pm.20.browseruk@xoxy.net

As is\, the following code cause a segfault​:

#! perl -slw use strict; use threads;

binmode STDIN\, '​:encoding(UTF-8)'; #binmode STDIN\, '​:raw';

async{ sleep 10; }->detach;

my $in = \;

Use binmode with any :encoding(*) on STDIN and it traps. Comment out either the uncommented binmode or the async lines and it no
longer traps. Comment out the encoding binmode and uncomment the :raw binmode and it no
longer traps. Comment out both binmodes and run with -CI on the command line and it no
longer traps.

Hence the conclusion that it is the Unicode IOlayers that are thread
hostile.

Also known to occur on linux systems.

Perl Info ``` Flags: category=core severity=critical Site configuration information for perl 5.10.1: Configured by sshd_server at Wed Jan 27 14:12:08 2010. Summary of my perl5 (revision 5 version 10 subversion 1) configuration: Platform: osname=MSWin32, osvers=5.2, archname=MSWin32-x64-multi-thread uname='' config_args='undef' hint=recommended, useposix=true, d_sigaction=undef useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=define, use64bitall=undef, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cl', ccflags ='-nologo -GF -W3 -MD -Zi -DNDEBUG -Ox -GL -Wp64 -fp:precise -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DWIN64 -DCONSERVATIVE -DUSE_SITECUSTOMIZE -DPRIVLIB_LAST_IN_INC -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX', optimize='-MD -Zi -DNDEBUG -Ox -GL -Wp64 -fp:precise', cppflags='-DWIN32' ccversion='15.0.21022', gccversion='', gccosandvers='' intsize=4, longsize=4, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=8 ivtype='__int64', ivsize=8, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"C:\Perl64\lib\CORE" -machine:AMD64' libpth=\lib libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib bufferoverflowU.lib msvcrt.lib perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib bufferoverflowU.lib msvcrt.lib libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl510.lib gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"C:\Perl64\lib\CORE" -machine:AMD64' Locally applied patches: ACTIVEPERL_LOCAL_PATCHES_ENTRY 0abd0d disable non-unicode case insensitive trie matching @INC for perl 5.10.1: C:/Perl64/site/lib C:/Perl64/lib . Environment for perl 5.10.1: HOME (unset) LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\Bin\amd64;c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\vcpackages;c:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE;C:\Program Files\Microsoft SDKs\Windows\v6.1\Bin\x64;C:\Program Files\Microsoft SDKs\Windows\v6.1\Bin;C:\Windows\Microsoft.NET\Framework64\v3.5;C:\Windows\Microsoft.NET\Framework\v3.5;C:\Windows\Microsoft.NET\Framework64\v2.0.50727;C:\Windows\Microsoft.NET\Framework\v2.0.50727;C:\Perl64\site\bin;C:\Perl64\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Program Files (x86)\AMD\CodeAnalyst\bin;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\lcc\bin PERL_BADLANG (unset) SHELL (unset) ```
p5pRT commented 12 years ago

From @Leont

On Wed Jan 18 16​:19​:08 2012\, pm.20.browseruk@​xoxy.net wrote​:

[Please describe your issue here]

As is\, the following code cause a segfault​:

#! perl -slw use strict; use threads;

binmode STDIN\, '​:encoding(UTF-8)'; #binmode STDIN\, '​:raw';

async{ sleep 10; }->detach;

my $in = \;

Use binmode with any :encoding(*) on STDIN and it traps. Comment out either the uncommented binmode or the async lines and it no longer traps. Comment out the encoding binmode and uncomment the :raw binmode and it no longer traps. Comment out both binmodes and run with -CI on the command line and it no longer traps.

Hence the conclusion that it is the Unicode IOlayers that are thread hostile.

Also known to occur on linux systems.

This bugreport is a duplicate of #31923

Leon

p5pRT commented 12 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 10 years ago

From @khwilliamson

On Wed Jan 18 18​:06​:29 2012\, LeonT wrote​:

On Wed Jan 18 16​:19​:08 2012\, pm.20.browseruk@​xoxy.net wrote​:

[Please describe your issue here]

As is\, the following code cause a segfault​:

#! perl -slw use strict; use threads;

binmode STDIN\, '​:encoding(UTF-8)'; #binmode STDIN\, '​:raw';

async{ sleep 10; }->detach;

my $in = \;

Use binmode with any :encoding(*) on STDIN and it traps. Comment out either the uncommented binmode or the async lines and it no longer traps. Comment out the encoding binmode and uncomment the :raw binmode and it no longer traps. Comment out both binmodes and run with -CI on the command line and it no longer traps.

Hence the conclusion that it is the Unicode IOlayers that are thread hostile.

Also known to occur on linux systems.

This bugreport is a duplicate of #31923

Leon

I am marking this as stalled\, as discussed in http​://markmail.org/message/kgbo6rasx4c7b3zw -- Karl Williamson

p5pRT commented 10 years ago

From [Unknown Contact. See original ticket]

On Wed Jan 18 18​:06​:29 2012\, LeonT wrote​:

On Wed Jan 18 16​:19​:08 2012\, pm.20.browseruk@​xoxy.net wrote​:

[Please describe your issue here]

As is\, the following code cause a segfault​:

#! perl -slw use strict; use threads;

binmode STDIN\, '​:encoding(UTF-8)'; #binmode STDIN\, '​:raw';

async{ sleep 10; }->detach;

my $in = \;

Use binmode with any :encoding(*) on STDIN and it traps. Comment out either the uncommented binmode or the async lines and it no longer traps. Comment out the encoding binmode and uncomment the :raw binmode and it no longer traps. Comment out both binmodes and run with -CI on the command line and it no longer traps.

Hence the conclusion that it is the Unicode IOlayers that are thread hostile.

Also known to occur on linux systems.

This bugreport is a duplicate of #31923

Leon

I am marking this as stalled\, as discussed in http​://markmail.org/message/kgbo6rasx4c7b3zw -- Karl Williamson

p5pRT commented 10 years ago

@khwilliamson - Status changed from 'open' to 'stalled'

p5pRT commented 9 years ago

From @jmaslak

Created by @jmaslak

This is a bug report for perl from jmaslak@​antelope.net\, generated with the help of perlbug 1.40 running under perl 5.22.0.

----------------------------------------------------------------- When setting STDOUT encoding to UTF-8\, in Perl 5.22.0\, a new thread creation will coredump.

Four line script to reproduce​:

use threads; binmode(STDOUT\, '​:encoding(UTF-8)'); threads->create(\&processthread); sub processthread { }

Result is a coredump at the threads->create line.

Perl Info ``` Flags: category=library severity=low module=threads Site configuration information for perl 5.22.0: Configured by jmaslak at Wed Jun 10 08:15:15 MDT 2015. Summary of my perl5 (revision 5 version 22 subversion 0) configuration: Platform: osname=linux, osvers=3.19.0-18-generic, archname=x86_64-linux-thread-multi uname='linux red 3.19.0-18-generic #18-ubuntu smp tue may 19 18:31:35 utc 2015 x86_64 x86_64 x86_64 gnulinux ' config_args='-de -Dprefix=/usr/local/perlbrew/perls/5.22.0.thread -Dusethreads -Dcccdlflags=-fPIC -Accflags=-fPIC -Aeval:scriptdir=/usr/local/perlbrew/perls/5.22.0.thread/bin' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fPIC -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2', cppflags='-D_REENTRANT -D_GNU_SOURCE -fPIC -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='', gccversion='4.9.2', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='cc', ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.21.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='2.21' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong' Locally applied patches: Devel::PatchPerl 1.34 @INC for perl 5.22.0: /usr/local/perlbrew/perls/5.22.0.thread/lib/site_perl/5.22.0/x86_64-linux-thread-multi /usr/local/perlbrew/perls/5.22.0.thread/lib/site_perl/5.22.0 /usr/local/perlbrew/perls/5.22.0.thread/lib/5.22.0/x86_64-linux-thread-multi /usr/local/perlbrew/perls/5.22.0.thread/lib/5.22.0 . Environment for perl 5.22.0: HOME=/home/jmaslak LANG=en_US.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH=:/usr/local/cuda-7.0/lib64 LOGDIR (unset) PATH=/usr/local/perlbrew/bin:/usr/local/perlbrew/perls/5.22.0.thread/bin:/home/jmaslak/.rakudobrew/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/cuda-7.0/bin PERLBREW_BASHRC_VERSION=0.58 PERLBREW_HOME=/home/jmaslak/.perlbrew PERLBREW_MANPATH=/usr/local/perlbrew/perls/5.22.0.thread/man PERLBREW_PATH=/usr/local/perlbrew/bin:/usr/local/perlbrew/perls/5.22.0.thread/bin PERLBREW_PERL=5.22.0.thread PERLBREW_ROOT=/usr/local/perlbrew PERLBREW_VERSION=0.58 PERL_BADLANG (unset) SHELL=/bin/bash ```
p5pRT commented 9 years ago

From @jkeenan

On Sat Jun 13 15​:22​:52 2015\, jmaslak@​antelope.net wrote​:

This is a bug report for perl from jmaslak@​antelope.net\, generated with the help of perlbug 1.40 running under perl 5.22.0.

----------------------------------------------------------------- When setting STDOUT encoding to UTF-8\, in Perl 5.22.0\, a new thread creation will coredump.

Four line script to reproduce​:

use threads; binmode(STDOUT\, '​:encoding(UTF-8)'); threads->create(\&processthread); sub processthread { }

Result is a coredump at the threads->create line.

Segfault confirmed at blead; see attached.

But this is not a new bug. I did threaded builds at tags 'v5.20.1' and 'v5.18.4' and got similar results. Attaching output for 5.18.4 threaded build.

-- James E Keenan (jkeenan@​cpan.org)

p5pRT commented 9 years ago

From @jkeenan

$ ./perl -Ilib -V Summary of my perl5 (revision 5 version 18 subversion 4) configuration​:   Commit id​: 51202371ba68f3f52f13124a3ea1bc3c171e0ee2   Platform​:   osname=linux\, osvers=3.13.0-54-generic\, archname=x86_64-linux-thread-multi   uname='linux zareason 3.13.0-54-generic #91-ubuntu smp tue may 26 19​:15​:08 utc 2015 x86_64 x86_64 x86_64 gnulinux '   config_args='-des -Dusedevel -Dusethreads'   hint=recommended\, useposix=true\, d_sigaction=define   useithreads=define\, usemultiplicity=define   useperlio=define\, d_sfio=undef\, uselargefiles=define\, usesocks=undef   use64bitint=define\, use64bitall=define\, uselongdouble=undef   usemymalloc=n\, bincompat5005=undef   Compiler​:   cc='cc'\, ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\,   optimize='-O2'\,   cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'   ccversion=''\, gccversion='4.8.2'\, gccosandvers=''   intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678   d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16   ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8   alignbytes=8\, prototype=define   Linker and Libraries​:   ld='cc'\, ldflags =' -fstack-protector -L/usr/local/lib'   libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib /lib64 /usr/lib64   libs=-lnsl -ldb -ldl -lm -lcrypt -lutil -lpthread -lc   perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc   libc=\, so=so\, useshrplib=false\, libperl=libperl.a   gnulibc_version='2.19'   Dynamic Linking​:   dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E'   cccdlflags='-fPIC'\, lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'

Characteristics of this binary (from libperl)​:   Compile-time options​: HAS_TIMES MULTIPLICITY PERLIO_LAYERS   PERL_DONT_CREATE_GVSV   PERL_HASH_FUNC_ONE_AT_A_TIME_HARD   PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP   PERL_PRESERVE_IVUV PERL_SAWAMPERSAND PERL_USE_DEVEL   USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS   USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE   USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_PERLIO   USE_PERL_ATOF USE_REENTRANT_API   Built under linux   Compiled at Jun 13 2015 21​:15​:59   %ENV​:   PERLBREW_BASHRC_VERSION="0.67"   PERLBREW_HOME="/home/jkeenan/.perlbrew"   PERLBREW_MANPATH="/home/jkeenan/perl5/perlbrew/perls/perl-5.22.0/man"   PERLBREW_PATH="/home/jkeenan/perl5/perlbrew/bin​:/home/jkeenan/perl5/perlbrew/perls/perl-5.22.0/bin"   PERLBREW_PERL="perl-5.22.0"   PERLBREW_ROOT="/home/jkeenan/perl5/perlbrew"   PERLBREW_VERSION="0.67"   PERL_WORKDIR="gitwork/perl"   @​INC​:   lib   /usr/local/lib/perl5/site_perl/5.18.4/x86_64-linux-thread-multi   /usr/local/lib/perl5/site_perl/5.18.4   /usr/local/lib/perl5/5.18.4/x86_64-linux-thread-multi   /usr/local/lib/perl5/5.18.4   .

$ ./perl -Ilib -Mthreads -e 'binmode(STDOUT\, "​:encoding(UTF-8)");threads->create(\&processthread);sub processthreads {};' Segmentation fault (core dumped)

p5pRT commented 9 years ago

From @jkeenan

[perl] 65 $ ./perl -Ilib -Mthreads -e 'binmode(STDOUT\, "​:encoding(UTF-8)");threads->create(\&processthread);sub processthreads {};' Segmentation fault (core dumped) [perl] 66 $ ./perl -Ilib -V Summary of my perl5 (revision 5 version 23 subversion 0) configuration​:   Commit id​: f10a29a7fe6e8572b69cb25659d584d3fd3dab5f   Platform​:   osname=linux\, osvers=3.13.0-54-generic\, archname=x86_64-linux-thread-multi   uname='linux zareason 3.13.0-54-generic #91-ubuntu smp tue may 26 19​:15​:08 utc 2015 x86_64 x86_64 x86_64 gnulinux '   config_args='-des -Dusedevel -Dusethreads'   hint=recommended\, useposix=true\, d_sigaction=define   useithreads=define\, usemultiplicity=define   use64bitint=define\, use64bitall=define\, uselongdouble=undef   usemymalloc=n\, bincompat5005=undef   Compiler​:   cc='cc'\, ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\,   optimize='-O2'\,   cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'   ccversion=''\, gccversion='4.8.2'\, gccosandvers=''   intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678\, doublekind=3   d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16\, longdblkind=3   ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8   alignbytes=8\, prototype=define   Linker and Libraries​:   ld='cc'\, ldflags =' -fstack-protector -L/usr/local/lib'   libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /lib64 /usr/lib64   libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc   perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc   libc=libc-2.19.so\, so=so\, useshrplib=false\, libperl=libperl.a   gnulibc_version='2.19'   Dynamic Linking​:   dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E'   cccdlflags='-fPIC'\, lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'

Characteristics of this binary (from libperl)​:   Compile-time options​: HAS_TIMES MULTIPLICITY PERLIO_LAYERS   PERL_DONT_CREATE_GVSV   PERL_HASH_FUNC_ONE_AT_A_TIME_HARD   PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP   PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV   PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT   USE_ITHREADS USE_LARGE_FILES USE_LOCALE   USE_LOCALE_COLLATE USE_LOCALE_CTYPE   USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO   USE_PERL_ATOF USE_REENTRANT_API   Built under linux   Compiled at Jun 13 2015 20​:06​:10   %ENV​:   PERLBREW_BASHRC_VERSION="0.67"   PERLBREW_HOME="/home/jkeenan/.perlbrew"   PERLBREW_MANPATH="/home/jkeenan/perl5/perlbrew/perls/perl-5.22.0/man"   PERLBREW_PATH="/home/jkeenan/perl5/perlbrew/bin​:/home/jkeenan/perl5/perlbrew/perls/perl-5.22.0/bin"   PERLBREW_PERL="perl-5.22.0"   PERLBREW_ROOT="/home/jkeenan/perl5/perlbrew"   PERLBREW_VERSION="0.67"   PERL_WORKDIR="gitwork/perl"   @​INC​:   lib   /usr/local/lib/perl5/site_perl/5.23.0/x86_64-linux-thread-multi   /usr/local/lib/perl5/site_perl/5.23.0   /usr/local/lib/perl5/5.23.0/x86_64-linux-thread-multi   /usr/local/lib/perl5/5.23.0   .

p5pRT commented 9 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 9 years ago

From @Leont

On Sun\, Jun 14\, 2015 at 12​:22 AM\, Joel Maslak \perlbug\-followup@&#8203;perl\.org wrote​:

When setting STDOUT encoding to UTF-8\, in Perl 5.22.0\, a new thread creation will coredump.

Four line script to reproduce​:

use threads; binmode(STDOUT\, '​:encoding(UTF-8)'); threads->create(\&processthread); sub processthread { }

Result is a coredump at the threads->create line.

This is a duplicate of #31923.

Leon

p5pRT commented 9 years ago

From @tonycoz

On Tue Dec 27 07​:50​:43 2011\, LeonT wrote​:

I seems calling methods during thread duplications is a bad idea\, so the easy solution would be to just not do that. Instead\, the argument should be saved in the PerlIOEncode struct\, and PerlIO_getargs should just return a duplicate of it.

That's easy enough\, but doesn't solve the problem.

PerlIOEncode_dup() calls PerlIOBase_dup() which calls PerlIOEncode_pushed()\, which then attempts to call Encode​::find_encoding() and fails again in the stack change.

I don't see an obvious way to tell _pushed() it's being called for a dup and to let the _dup() handler finish the job.

I'm not too sure about memory allocation for the PerlIO structures either\, but that's unrelated to this particular issue.

Tony

p5pRT commented 9 years ago

The RT System itself - Status changed from 'stalled' to 'open'

p5pRT commented 9 years ago

From @Leont

On Wed\, Jun 17\, 2015 at 2​:30 AM\, Tony Cook via RT \<perlbug-followup@​perl.org

wrote​:

On Tue Dec 27 07​:50​:43 2011\, LeonT wrote​:

I seems calling methods during thread duplications is a bad idea\, so the easy solution would be to just not do that. Instead\, the argument should be saved in the PerlIOEncode struct\, and PerlIO_getargs should just return a duplicate of it.

That's easy enough\, but doesn't solve the problem.

PerlIOEncode_dup() calls PerlIOBase_dup() which calls PerlIOEncode_pushed()\, which then attempts to call Encode​::find_encoding() and fails again in the stack change.

Yeah\, I had since reached the same conclusion\, otherwise I would have fixed this already.

I don't see an obvious way to tell _pushed() it's being called for a dup and to let the _dup() handler finish the job.

I suspect that allocating a new layer\, and pushing it directly is the only way out. Precedent for this is lacking though.

I'm not too sure about memory allocation for the PerlIO structures either\, but that's unrelated to this particular issue.

AFAIK allocating memory is safe.

Leon

p5pRT commented 9 years ago

From perl@profvince.com

I've pushed a tentative fix for this into the vincent/thread-safe-encoding branch.

Tony\, Leon\, care to have a look at this?

Vincent

p5pRT commented 9 years ago

From perl@profvince.com

0001-Properly-duplicate-PerlIO-encoding-objects.patch ```diff From 1f20b10901126aaa2e568735d65ac32b7ba28ca7 Mon Sep 17 00:00:00 2001 From: Vincent Pit Date: Fri, 28 Aug 2015 14:17:00 -0300 Subject: [PATCH] Properly duplicate PerlIO::encoding objects PerlIO::encoding objects are usually initialized by calling Perl methods, essentially from the pushed() and getarg() callbacks. During cloning, the PerlIO API will by default call these methods to initialize the duplicate struct when the PerlIOBase parent struct is itself duplicated. This does not behave so well because the perl interpreter is not ready to call methods at this point, for the stacks are not set up yet. The proper way to duplicate the PerlIO::encoding object is to call sv_dup() on its members from the dup() PerlIO callback. So the only catch is to make the getarg() and pushed() calls implied by the duplication of the underlying PerlIOBase object aware that they are called during cloning, and make them wait that the control flow returns to the dup() callback. Fortunately, getarg() knows since its param argument is then non-null, and its return value is passed immediately to pushed(), so it is enough to tag this returned value with a custom magic so that pushed() can see it is being called during cloning. This fixes [RT #31923]. --- ext/PerlIO-encoding/encoding.pm | 2 +- ext/PerlIO-encoding/encoding.xs | 25 +++++++++++++++++++++++-- 2 files changed, 24 insertions(+), 3 deletions(-) diff --git a/ext/PerlIO-encoding/encoding.pm b/ext/PerlIO-encoding/encoding.pm index 4cff76d..97f05ec 100644 --- a/ext/PerlIO-encoding/encoding.pm +++ b/ext/PerlIO-encoding/encoding.pm @@ -1,7 +1,7 @@ package PerlIO::encoding; use strict; -our $VERSION = '0.21'; +our $VERSION = '0.22'; our $DEBUG = 0; $DEBUG and warn __PACKAGE__, " called by ", join(", ", caller), "\n"; diff --git a/ext/PerlIO-encoding/encoding.xs b/ext/PerlIO-encoding/encoding.xs index 03b8850..c992dd2 100644 --- a/ext/PerlIO-encoding/encoding.xs +++ b/ext/PerlIO-encoding/encoding.xs @@ -49,13 +49,23 @@ typedef struct { #define NEEDS_LINES 1 +static MGVTBL PerlIOEncode_tag = { 0, 0, 0, 0, 0, 0, 0, 0 }; + SV * PerlIOEncode_getarg(pTHX_ PerlIO * f, CLONE_PARAMS * param, int flags) { PerlIOEncode *e = PerlIOSelf(f, PerlIOEncode); - SV *sv = &PL_sv_undef; - PERL_UNUSED_ARG(param); + SV *sv; PERL_UNUSED_ARG(flags); + /* During cloning, return an undef token object so that _pushed() knows + * that it should not call methods and wait for _dup() to actually dup the + * encoding object. */ + if (param) { + sv = newSV(0); + sv_magicext(sv, NULL, PERL_MAGIC_ext, &PerlIOEncode_tag, 0, 0); + return sv; + } + sv = &PL_sv_undef; if (e->enc) { dSP; /* Not 100% sure stack swap is right thing to do during dup ... */ @@ -85,6 +95,14 @@ PerlIOEncode_pushed(pTHX_ PerlIO * f, const char *mode, SV * arg, PerlIO_funcs * IV code = PerlIOBuf_pushed(aTHX_ f, mode, Nullsv,tab); SV *result = Nullsv; + if (SvTYPE(arg) >= SVt_PVMG + && mg_findext(arg, PERL_MAGIC_ext, &PerlIOEncode_tag)) { + e->enc = NULL; + e->chk = NULL; + e->inEncodeCall = 0; + return code; + } + PUSHSTACKi(PERLSI_MAGIC); ENTER; SAVETMPS; @@ -566,6 +584,9 @@ PerlIOEncode_dup(pTHX_ PerlIO * f, PerlIO * o, if (oe->enc) { fe->enc = PerlIO_sv_dup(aTHX_ oe->enc, params); } + if (oe->chk) { + fe->chk = PerlIO_sv_dup(aTHX_ oe->chk, params); + } } return f; } -- 1.9.5 (Apple Git-50.3) ```
p5pRT commented 9 years ago

From @tonycoz

On Fri Aug 28 11​:54​:03 2015\, perl@​profvince.com wrote​:

I've pushed a tentative fix for this into the vincent/thread-safe-encoding branch.

Tony\, Leon\, care to have a look at this?

Makes sense to me.

Tony

p5pRT commented 9 years ago

From bitcard@profvince.com

Now in blead as commit 0ee3fa26. Closing.

p5pRT commented 9 years ago

bitcard@profvince.com - Status changed from 'open' to 'resolved'

bram-perl commented 2 years ago

Searchable as RT31923$

Also: RT86550$

(There is a reference in the code to [perl #86550], that ticket - was apparently - merged into this RT31923 but nothing in this ticket contained the number 86650 so adding it now to make a future search for that ticket easier. Start of RT86650 was: https://github.com/Perl/perl5/issues/7533#issuecomment-543950068 and/or https://www.nntp.perl.org/group/perl.perl5.porters/2011/03/msg170204.html )