segfault (dead loop) with Encoding, use open :locale, print STDERR

p5pRT commented 17 years ago

Migrated from rt.perl.org#41442 (status was 'resolved')

Searchable as RT41442$

p5pRT commented 20 years ago

From aa29@mail.ru

To: perlbug@perl.org Subject: [PerlIO] Warning with char that does not map to STDERR encoding leads to infinite cycle. Reply-To: aa29@mail.ru Message-Id: \5\.8\.4\_3176\_1084980326@INFORMED

This is a bug report for perl from aa29@aa29xp\, generated with the help of perlbug 1.35 running under perl v5.8.4.

PerlIO: Warning with char that does not map to STDERR encoding leads to infinite cycle.

binmode(STDERR\, ":encoding(cp866)"); warn "\302\253"; # no char in cp866 for this

Then Perl tries to encode to cp866 while flushing STDERR buffer\, it finds out that this char cannot be mapped. Default check-mode is ENCODE_WARN_ON_ERR | ENCODE_PERLQQ\, so it generates warning:

ext\Encode\Encode.xs\, line 153:

if (check & ENCODE_WARN_ON_ERR){ Perl_warner(aTHX_ packWARN(WARN_UTF8)\, ERR_ENCODE_NOMAP\, (UV)ch\, enc->name[0]);

But "\302\253" already is in the STDERR buffer\, and Perl_warner just adds ERR_ENCODE_NOMAP to this buffer and flushes\, and then it comes right to the same line\, and so on.

Flags: category=core severity=medium

Site configuration information for perl v5.8.4:

Configured by aa29 at Mon May 17 17:59:46 2004.

Summary of my perl5 (revision 5 version 8 subversion 4) configuration: Platform: osname=MSWin32\, osvers=4.0\, archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended\, useposix=true\, d_sigaction=undef usethreads=undef use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='cl'\, ccflags ='-nologo -Gf -W3 -MD -Zi -DNDEBUG -O1 -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE _DES_FCRYPT -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL _MSVCRT_READFIX'\, optimize='-MD -Zi -DNDEBUG -O1'\, cppflags='-DWIN32' ccversion=''\, gccversion=''\, gccosandvers='' intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234 d_longlong=undef\, longlongsize=8\, d_longdbl=define\, longdblsize=10 ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='__int64'\, lseeksize=8 alignbytes=8\, prototype=define Linker and Libraries: ld='link'\, ldflags '-nologo -nodefaultlib -debug -opt:ref\,icf -libpath:"c:\perl\lib\CORE" -ma chine:x86' libpth=\lib libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib msvcrt.lib perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib msvcrt.lib libc=msvcrt.lib\, so=dll\, useshrplib=yes\, libperl=perl58.lib gnulibc_version='undef' Dynamic Linking: dlsrc=dl_win32.xs\, dlext=dll\, d_dlsymun=undef\, ccdlflags=' ' cccdlflags=' '\, lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref\,icf -libpath:"c:\perl \lib\CORE" -machine:x86'

Locally applied patches:

@INC for perl v5.8.4: C:/Perl/lib C:/Perl/site/lib .

Environment for perl v5.8.4: HOME (unset) LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset)

PATH=C:\cygwin\bin;C:\Tcl\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\Syst em32\Wbem;C:\Perl\bin;C:\Program Files\Support Tools;D:\src\lib;C:\Program Files\Microsoft Visual Studio\Common\Tools\WinNT;C:\Program Files\Microsoft Visual Studio\Common\MSDev98\Bin;C:\Program Files\Microsoft Visual Studio\Common\Tools;C:\Program Files\Microsoft Visual Studio\VC98\bin;C:\Arc;C:\Program Files\Utils;C:\Mysql\bin;C:\Program Files\Debugging Tools for Windows;C:\Tcl\bin;D:\Linda\XML\fop;C:\Program Files\GNU\WinCvs 1.2;D:\src\bin;C:\Program Files\Far PERL_BADLANG (unset) SHELL (unset)

aa29

p5pRT commented 19 years ago

From bulb@ucw.cz

Created by bulb@ucw.cz

Hello\,

I have recently seen perl crashing with segfault for certain combinations of locale and output. I have finally managed to narrow the problem down to the following snippet (note\, that it is utf-8 encoded and contains a non-iso-8859-* character):

#!/usr/bin/perl use utf8; $string = "è¶\n"; binmode STDERR\, ":encoding(iso-8859-2)"; print STDERR $string; __END__

This script crashes on 3 different systems. One is SuSE 9.0 with perl 5.8.1\, another is Debian Etch on i686 with perl 5.8.7 and this one which is Debian Sid on amd64 with perl 5.8.7.

The script does NOT crash when STDERR is replaced by STDOUT\, but rather prints:

"\x{8336}" does not map to iso-8859-2 at perlio-bug.pl line 9. \x{8336}

as expected. Since the contents of the perlio layer stack might be important here\, these are dumps of default perlio stacs (with commands that produced them):

bulb@efreet:\~$ perl -MData::Dumper -MPerlIO -e 'print Dumper(PerlIO::get_layers(STDERR\, details => 1))' $VAR1 = 'unix'; $VAR2 = undef; $VAR3 = 13177344; $VAR4 = 'perlio'; $VAR5 = undef; $VAR6 = 13177344;

bulb@efreet:\~$ perl -MData::Dumper -MPerlIO -e 'print Dumper(PerlIO::get_layers(STDOUT\, details => 1))' $VAR1 = 'unix'; $VAR2 = undef; $VAR3 = 13111808; $VAR4 = 'perlio'; $VAR5 = undef; $VAR6 = 13111808;

bulb@efreet:\~$ perl -MData::Dumper -MPerlIO -e 'binmode(STDERR\, ":encoding(iso-8859-2)"); print Dumper(PerlIO::get_layers(STDERR\, details => 1))' $VAR1 = 'unix'; $VAR2 = undef; $VAR3 = 13144576; $VAR4 = 'perlio'; $VAR5 = undef; $VAR6 = 13144576; $VAR7 = 'encoding'; $VAR8 = 'iso-8859-2'; $VAR9 = 13144576;

bulb@efreet:\~$ perl -MData::Dumper -MPerlIO -e 'binmode(STDOUT\, ":encoding(iso-8859-2)"); print Dumper(PerlIO::get_layers(STDOUT\, details => 1))' $VAR1 = 'unix'; $VAR2 = undef; $VAR3 = 13144576; $VAR4 = 'perlio'; $VAR5 = undef; $VAR6 = 13144576; $VAR7 = 'encoding'; $VAR8 = 'iso-8859-2'; $VAR9 = 13144576;

This is a distribution perl\, which is stripped\, so the backtrace won't be exactly useful\, but here it is (or rather part of it) anyway:

(gdb) bt #0 0x00002aaaab20c844 in realloc () from /lib/libc.so.6 #1 0x00002aaaaac3f68f in Perl_safesysrealloc () from /usr/lib/libperl.so.5.8 #2 0x00002aaaaac5ed8b in Perl_sv_grow () from /usr/lib/libperl.so.5.8 #3 0x00002aaaaac6c2ce in Perl_sv_catpvn_flags () from /usr/lib/libperl.so.5.8 #4 0x00002aaaaac69807 in Perl_sv_vcatpvfn () from /usr/lib/libperl.so.5.8 #5 0x00002aaaaac409f8 in Perl_vmess () from /usr/lib/libperl.so.5.8 #6 0x00002aaaaac41955 in Perl_vwarn () from /usr/lib/libperl.so.5.8 #7 0x00002aaaaac41fd3 in Perl_vwarner () from /usr/lib/libperl.so.5.8 #8 0x00002aaaaac42183 in Perl_warner () from /usr/lib/libperl.so.5.8 #9 0x00002aaaab8125cd in call_failure () from /usr/lib/perl/5.8/auto/Encode/Encode.so #10 0x00002aaaab813a27 in XS_Encode__XS_encode () from /usr/lib/perl/5.8/auto/Encode/Encode.so #11 0x00002aaaaac59625 in Perl_pp_entersub () from /usr/lib/libperl.so.5.8 #12 0x00002aaaaac51b0e in Perl_runops_standard () from /usr/lib/libperl.so.5.8 #13 0x00002aaaaabf90c8 in Perl_call_sv () from /usr/lib/libperl.so.5.8 #14 0x00002aaaab70d285 in PerlIOEncode_flush () from /usr/lib/perl/5.8/auto/PerlIO/encoding/encoding.so #15 0x00002aaaaacbab5f in Perl_PerlIO_flush () from /usr/lib/libperl.so.5.8 #16 0x00002aaaaacbf0ed in PerlIOBuf_write () from /usr/lib/libperl.so.5.8 #17 0x00002aaaaac40d1e in Perl_write_to_stderr () from /usr/lib/libperl.so.5.8 #18 0x00002aaaaac41baf in Perl_vwarn () from /usr/lib/libperl.so.5.8 #19 0x00002aaaaac41fd3 in Perl_vwarner () from /usr/lib/libperl.so.5.8 #20 0x00002aaaaac42183 in Perl_warner () from /usr/lib/libperl.so.5.8 #21 0x00002aaaab8125cd in call_failure () from /usr/lib/perl/5.8/auto/Encode/Encode.so #22 0x00002aaaab813a27 in XS_Encode__XS_encode () from /usr/lib/perl/5.8/auto/Encode/Encode.so #23 0x00002aaaaac59625 in Perl_pp_entersub () from /usr/lib/libperl.so.5.8 #24 0x00002aaaaac51b0e in Perl_runops_standard () from /usr/lib/libperl.so.5.8 #25 0x00002aaaaabf90c8 in Perl_call_sv () from /usr/lib/libperl.so.5.8 #26 0x00002aaaab70d285 in PerlIOEncode_flush () from /usr/lib/perl/5.8/auto/PerlIO/encoding/encoding.so #27 0x00002aaaaacbab5f in Perl_PerlIO_flush () from /usr/lib/libperl.so.5.8 #28 0x00002aaaaacbf0ed in PerlIOBuf_write () from /usr/lib/libperl.so.5.8 #29 0x00002aaaaac40d1e in Perl_write_to_stderr () from /usr/lib/libperl.so.5.8 #30 0x00002aaaaac41baf in Perl_vwarn () from /usr/lib/libperl.so.5.8 #31 0x00002aaaaac41fd3 in Perl_vwarner () from /usr/lib/libperl.so.5.8 #32 0x00002aaaaac42183 in Perl_warner () from /usr/lib/libperl.so.5.8 #33 0x00002aaaab8125cd in call_failure () from /usr/lib/perl/5.8/auto/Encode/Encode.so #34 0x00002aaaab813a27 in XS_Encode__XS_encode () from /usr/lib/perl/5.8/auto/Encode/Encode.so #35 0x00002aaaaac59625 in Perl_pp_entersub () from /usr/lib/libperl.so.5.8 #36 0x00002aaaaac51b0e in Perl_runops_standard () from /usr/lib/libperl.so.5.8 #37 0x00002aaaaabf90c8 in Perl_call_sv () from /usr/lib/libperl.so.5.8 #38 0x00002aaaab70d285 in PerlIOEncode_flush () from /usr/lib/perl/5.8/auto/PerlIO/encoding/encoding.so #39 0x00002aaaaacbab5f in Perl_PerlIO_flush () from /usr/lib/libperl.so.5.8 #40 0x00002aaaaacbf0ed in PerlIOBuf_write () from /usr/lib/libperl.so.5.8 #41 0x00002aaaaac40d1e in Perl_write_to_stderr () from /usr/lib/libperl.so.5.8 #42 0x00002aaaaac41baf in Perl_vwarn () from /usr/lib/libperl.so.5.8 #43 0x00002aaaaac41fd3 in Perl_vwarner () from /usr/lib/libperl.so.5.8 #44 0x00002aaaaac42183 in Perl_warner () from /usr/lib/libperl.so.5.8 #45 0x00002aaaab8125cd in call_failure () from /usr/lib/perl/5.8/auto/Encode/Encode.so #46 0x00002aaaab813a27 in XS_Encode__XS_encode () from /usr/lib/perl/5.8/auto/Encode/Encode.so #47 0x00002aaaaac59625 in Perl_pp_entersub () from /usr/lib/libperl.so.5.8 #48 0x00002aaaaac51b0e in Perl_runops_standard () from /usr/lib/libperl.so.5.8 #49 0x00002aaaaabf90c8 in Perl_call_sv () from /usr/lib/libperl.so.5.8 #50 0x00002aaaab70d285 in PerlIOEncode_flush () from /usr/lib/perl/5.8/auto/PerlIO/encoding/encoding.so #51 0x00002aaaaacbab5f in Perl_PerlIO_flush () from /usr/lib/libperl.so.5.8 #52 0x00002aaaaacbf0ed in PerlIOBuf_write () from /usr/lib/libperl.so.5.8 #53 0x00002aaaaac40d1e in Perl_write_to_stderr () from /usr/lib/libperl.so.5.8 #54 0x00002aaaaac41baf in Perl_vwarn () from /usr/lib/libperl.so.5.8 #55 0x00002aaaaac41fd3 in Perl_vwarner () from /usr/lib/libperl.so.5.8 #56 0x00002aaaaac42183 in Perl_warner () from /usr/lib/libperl.so.5.8 #57 0x00002aaaab8125cd in call_failure () from /usr/lib/perl/5.8/auto/Encode/Encode.so #58 0x00002aaaab813a27 in XS_Encode__XS_encode () from /usr/lib/perl/5.8/auto/Encode/Encode.so

And this goes on for THOUSANDS of lines. It ends LOOOONG way down by:

#66106 0x00002aaaab813a27 in XS_Encode__XS_encode () from /usr/lib/perl/5.8/auto/Encode/Encode.so #66107 0x00002aaaaac59625 in Perl_pp_entersub () from /usr/lib/libperl.so.5.8 #66108 0x00002aaaaac51b0e in Perl_runops_standard () from /usr/lib/libperl.so.5.8 #66109 0x00002aaaaabf90c8 in Perl_call_sv () from /usr/lib/libperl.so.5.8 #66110 0x00002aaaab70d285 in PerlIOEncode_flush () from /usr/lib/perl/5.8/auto/PerlIO/encoding/encoding.so #66111 0x00002aaaaacbab5f in Perl_PerlIO_flush () from /usr/lib/libperl.so.5.8 #66112 0x00002aaaaacbf073 in PerlIOBuf_write () from /usr/lib/libperl.so.5.8 #66113 0x00002aaaaaca2b4f in Perl_do_print () from /usr/lib/libperl.so.5.8 #66114 0x00002aaaaac534c8 in Perl_pp_print () from /usr/lib/libperl.so.5.8 #66115 0x00002aaaaac51b0e in Perl_runops_standard () from /usr/lib/libperl.so.5.8 #66116 0x00002aaaaabf9dbb in perl_run () from /usr/lib/libperl.so.5.8 #66117 0x000000000040180c in main ()

Feel free to ask for more information.

Regards\,

Jan Hudec \bulb@ucw\.cz

Perl Info

``` Flags: category=core severity=medium Site configuration information for perl v5.8.7: Configured by Debian Project at Tue Sep 13 02:36:46 CEST 2005. Summary of my perl5 (revision 5 version 8 subversion 7) configuration: Platform: osname=linux, osvers=2.6.12-1-amd64-k8, archname=x86_64-linux-gnu-thread-multi uname='linux athlon.lowpingbastards.de 2.6.12-1-amd64-k8 #1 sun jul 17 15:04:25 cest 2005 x86_64 gnulinux ' config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.8.7 -Dsitearch=/usr/local/lib/perl/5.8.7 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.7 -Dd_dosuid -des' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2', cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include' ccversion='', gccversion='4.0.2 20050821 (prerelease) (Debian 4.0.1-6)', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt perllibs=-ldl -lm -lpthread -lc -lcrypt libc=/lib/libc-2.3.5.so, so=so, useshrplib=true, libperl=libperl.so.5.8.7 gnulibc_version='2.3.5' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib' Locally applied patches: @INC for perl v5.8.7: /home/bulb/lib/perl/5.8.7 /home/bulb/share/perl/5.8.7 /etc/perl /usr/local/lib/perl/5.8.7 /usr/local/share/perl/5.8.7 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.8 /usr/share/perl/5.8 /usr/local/lib/site_perl . Environment for perl v5.8.7: HOME=/home/bulb LANG (unset) LANGUAGE=en_GB:en_US:en:cs_CZ LC_CTYPE=cs_CZ.UTF-8 LC_TIME=cs_CZ.UTF-8 LD_LIBRARY_PATH=/home/bulb/lib/ LOGDIR (unset) PATH=/home/bulb/bin:/home/bulb/shellstuff//bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games PERL5LIB=/home/bulb/lib/perl/5.8.7:/home/bulb/share/perl/5.8.7 PERL_BADLANG (unset) SHELL=/bin/zsh-beta ```

p5pRT commented 19 years ago

From bulb@ucw.cz

Hello Again\,

I've played with the bug a little more and I include several more scripts\, that should help pinning down the bug.

First\, let's get rid of the dependence on UTF-8 in the mail. This is the original testcase\, with escape sequence instead of the non-ascii character. Also it now says :encoding(ascii) instead of iso-8859-2.

#!/usr/bin/perl use utf8; $string = "\x{8336}\n"; binmode STDERR\, ":encoding(ascii)"; print STDERR $string; __END__

Now the next script does NOT crash. It works normally:

#!/usr/bin/perl use utf8; $SIG{__WARN__} = sub { }; $string = "\x{8336}\n"; binmode STDERR\, ":encoding(ascii)"; print STDERR $string; __END__

As you can see\, the only difference is the line:

$SIG{__WARN__} = sub { };

So it shows\, that the problem is caused by printing to STDERR from a warning generated inside print to STDERR. The next script crashes again:

#!/usr/bin/perl use utf8; $SIG{__WARN__} = sub { print STDERR "BUG\n"; }; $string = "\x{8336}\n"; binmode STDERR\, ":encoding(ascii)"; print STDERR $string; __END__

Now if the print is not to STDERR\, it works. The following script does not crash:

#!/usr/bin/perl use utf8; $SIG{__WARN__} = sub { print "BUG\n"; }; $string = "\x{8336}\n"; binmode STDERR\, ":encoding(ascii)"; print STDERR $string; __END__

The most surprising is this script though. The next script replaces STDERR with STDOUT:

#!/usr/bin/perl use utf8; $SIG{__WARN__} = sub { print STDOUT "BUG\n"; }; $string = "\x{8336}\n"; binmode STDOUT\, ":encoding(ascii)"; print STDOUT $string; __END__

Now this script does crash\, but in a DIFFERENT WAY. Instead of immediately dying of SIGSEGV\, it prints the following:

"\x{8336}" does not map to ascii at perlio-bug4.pl line 3. \x{8336} BUG panic: sv_setpvn called with negative strlen at perlio-bug4.pl line 6.

And exits with status 2. Note\, that this was on a terminal\, where STDOUT is line-buffered (but not unbuffered).

-- Jan 'Bulb' Hudec \bulb@ucw\.cz

p5pRT commented 19 years ago

From @smpeters

[bulb@ucw.cz - Fri Sep 23 10:44:07 2005]:

Hello Again\,

I've played with the bug a little more and I include several more scripts\, that should help pinning down the bug.

First\, let's get rid of the dependence on UTF-8 in the mail. This is the original testcase\, with escape sequence instead of the non-ascii character. Also it now says :encoding(ascii) instead of iso-8859-2.

#!/usr/bin/perl use utf8; $string = "\x{8336}\n"; binmode STDERR\, ":encoding(ascii)"; print STDERR $string; __END__

Now the next script does NOT crash. It works normally:

#!/usr/bin/perl use utf8; $SIG{__WARN__} = sub { }; $string = "\x{8336}\n"; binmode STDERR\, ":encoding(ascii)"; print STDERR $string; __END__

As you can see\, the only difference is the line:

$SIG{__WARN__} = sub { };

So it shows\, that the problem is caused by printing to STDERR from a warning generated inside print to STDERR. The next script crashes again:

#!/usr/bin/perl use utf8; $SIG{__WARN__} = sub { print STDERR "BUG\n"; }; $string = "\x{8336}\n"; binmode STDERR\, ":encoding(ascii)"; print STDERR $string; __END__

Now if the print is not to STDERR\, it works. The following script does not crash:

#!/usr/bin/perl use utf8; $SIG{__WARN__} = sub { print "BUG\n"; }; $string = "\x{8336}\n"; binmode STDERR\, ":encoding(ascii)"; print STDERR $string; __END__

The most surprising is this script though. The next script replaces STDERR with STDOUT:

#!/usr/bin/perl use utf8; $SIG{__WARN__} = sub { print STDOUT "BUG\n"; }; $string = "\x{8336}\n"; binmode STDOUT\, ":encoding(ascii)"; print STDOUT $string; __END__

Now this script does crash\, but in a DIFFERENT WAY. Instead of immediately dying of SIGSEGV\, it prints the following:

"\x{8336}" does not map to ascii at perlio-bug4.pl line 3. \x{8336} BUG panic: sv_setpvn called with negative strlen at perlio-bug4.pl line 6.

And exits with status 2. Note\, that this was on a terminal\, where STDOUT is line-buffered (but not unbuffered).

The code is essentially going into an infinite loop. The code discovers the problem with the encoding and when attempting to flush the IO in ext/PerlIO/encoding/encoding.xs calls Encode::encode()\, starting the cycle all over again. Since Perl_warner() is not capable of handling anything but char* data\, is there a way to turn off encoding while it warns\, then turn it back on? A sort of "no utf-8;" within the core code itself?

Here's the fatal end of the stack trace.

#0 0x080fc619 in Perl_sv_vcatpvfn (sv=0x933e5d8\, pat=0xb7f6a348 "\"\\x{%04lx}\" does not map to %s"\, patlen=30\, args=0xbf184324\, svargs=0x0\, svmax=0\, maybe_tainted=0x0) at sv.c:8832 8832 bool has_utf8 = DO_UTF8(sv); /* has the result utf8? */ (gdb) bt #0 0x080fc619 in Perl_sv_vcatpvfn (sv=0x933e5d8\, pat=0xb7f6a348 "\"\\x{%04lx}\" does not map to %s"\, patlen=30\, args=0xbf184324\, svargs=0x0\, svmax=0\, maybe_tainted=0x0) at sv.c:8832 #1 0x080fc425 in Perl_sv_vsetpvfn (sv=0x933e5d8\, pat=0xb7f6a348 "\"\\x{%04lx}\" does not map to %s"\, patlen=30\, args=0xbf184324\, svargs=0x0\, svmax=0\, maybe_tainted=0x0) at sv.c:8754 #2 0x080c7d3e in Perl_vmess ( pat=0xb7f6a348 "\"\\x{%04lx}\" does not map to %s"\, args=0xbf184324) at util.c:991 #3 0x080c8c89 in Perl_vwarn ( pat=0xb7f6a348 "\"\\x{%04lx}\" does not map to %s"\, args=0xbf184324) at util.c:1260 #4 0x080c943f in Perl_vwarner (err=44\, pat=0xb7f6a348 "\"\\x{%04lx}\" does not map to %s"\, args=0xbf184324) at util.c:1377 #5 0x080c9199 in Perl_warner (err=44\, pat=0xb7f6a348 "\"\\x{%04lx}\" does not map to %s") at util.c:1350 #6 0xb7f65537 in encode_method (enc=0xb7c749d8\, dir=0xb7c713e0\, src=0x819e0c8\, check=2306\, offset=0x0\, term=0x0\, retcode=0x0) at Encode.xs:162 #7 0xb7f6735b in XS_Encode__XS_encode (cv=0x821f548) at Encode.xs:564 #8 0x080e8bbf in Perl_pp_entersub () at pp_hot.c:2788 #9 0x080c5d55 in Perl_runops_debug () at dump.c:1597 #10 0x08064d6b in S_call_body (myop=0xbf184640\, is_eval=0 '\0') at perl.c:2617 #11 0x080645d7 in Perl_call_sv (sv=0x933e5b8\, flags=64) at perl.c:2515 #12 0x0806433a in Perl_call_method (methname=0xb7c7a5b0 "encode"\, flags=0) at perl.c:2448 #13 0xb7c78e60 in PerlIOEncode_flush (f=0x81abd24) at encoding.xs:420 #14 0x08164983 in Perl_PerlIO_flush (f=0x81abd24) at perlio.c:1615 #15 0x081678c5 in PerlIOBuf_write (f=0x81abd24\, vbuf=0x935ef00\, count=61) at perlio.c:3776 #16 0xb7c79bca in PerlIOEncode_write (f=0x81abd24\, vbuf=0x935ef00\, count=61) at encoding.xs:584 #17 0x08164826 in Perl_PerlIO_write (f=0x81abd24\, vbuf=0x935ef00\, count=61) at perlio.c:1592 #18 0x080c845c in Perl_write_to_stderr ( message=0x935ef00 "\"\\x{00e8}\" does not map to iso-8859-2 at rt_37234.pl line 6.\n"\, msgl en=61) at util.c:1060 #19 0x080c9153 in Perl_vwarn ( pat=0xb7f6a348 "\"\\x{%04lx}\" does not map to %s"\, args=0xbf184944) at util.c:1300 #20 0x080c943f in Perl_vwarner (err=44\, pat=0xb7f6a348 "\"\\x{%04lx}\" does not map to %s"\, args=0xbf184944) at util.c:1377

p5pRT commented 19 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 17 years ago

From pixel@mandriva.com

This is a bug report for perl from pixel@mandriva.com\, generated with the help of perlbug 1.35 running under perl v5.8.8.

this segfaults if the locale charset is not utf8:

use open ':locale'; print STDERR "\x{201e}"; # „

Flags: category=core severity=medium

Site configuration information for perl v5.8.8:

Configured by Mandriva at Fri Sep 8 20:00:54 CEST 2006.

Summary of my perl5 (revision 5 version 8 subversion 8) configuration: Platform: osname=linux\, osvers=2.6.12-12mdksmp\, archname=i386-linux uname='linux n4.mandriva.com 2.6.12-12mdksmp #1 smp fri sep 9 17:43:23 cest 2005 i686 intel(r) xeon(tm) cpu 2.80ghz gnulinux ' config_args='-des -Dinc_version_list=5.8.7 5.8.7/i386-linux 5.8.6 5.8.6/i386-linux 5.8.5 5.8.4 5.8.3 5.8.2 5.8.1 5.8.0 5.6.1 5.6.0 -Darchname=i386-linux -Dcc=gcc -Doptimize=-O2 -pipe -Wp\,-D_FORTIFY_SOURCE=2 -fexceptions -fomit-frame-pointer -march=i586 -mtune=pentiumpro -fasynchronous-unwind-tables -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr -Dsitebin=/usr/local/bin -Dsiteman1dir=/usr/local/share/man/man1 -Dsiteman3dir=/usr/local/share/man/man3 -Dman3ext=3pm -Dcf_by=Mandriva -Dmyhostname=localhost -Dperladmin=root@localhost -Dcf_email=root@localhost -Dd_dosuid -Ud_csh -Duseshrplib' hint=recommended\, useposix=true\, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='gcc'\, ccflags ='-fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm'\, optimize='-O2 -pipe -Wp\,-D_FORTIFY_SOURCE=2 -fexceptions -fomit-frame-pointer -march=i586 -mtune=pentiumpro -fasynchronous-unwind-tables'\, cppflags='-fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include -I/usr/include/gdbm' ccversion=''\, gccversion='4.1.1 20060724 (prerelease) (4.1.1-3mdk)'\, gccosandvers='' intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=12 ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=4\, prototype=define Linker and Libraries: ld='gcc'\, ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lnsl -lndbm -lgdbm -ldl -lm -lcrypt -lutil -lc perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc libc=/lib/libc-2.4.so\, so=so\, useshrplib=true\, libperl=libperl.so gnulibc_version='2.4' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E -Wl\,-rpath\,/usr/lib/perl5/5.8.8/i386-linux/CORE' cccdlflags='-fPIC'\, lddlflags='-shared -L/usr/local/lib'

Locally applied patches: Mandriva Linux patches

@INC for perl v5.8.8: /home/pixel/etc/perl /home/pixel/etc/perl/i386-linux /home/pixel/etc/perl/site_perl /usr/lib/perl5/5.8.8/i386-linux /usr/lib/perl5/5.8.8 /usr/lib/perl5/site_perl/5.8.8/i386-linux /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.8/i386-linux /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl/5.8.7 /usr/lib/perl5/vendor_perl/5.8.7/i386-linux /usr/lib/perl5/vendor_perl/5.8.6 /usr/lib/perl5/vendor_perl/5.8.6/i386-linux /usr/lib/perl5/vendor_perl/5.8.5 /usr/lib/perl5/vendor_perl/5.8.4 /usr/lib/perl5/vendor_perl/5.8.3 /usr/lib/perl5/vendor_perl/5.8.2 /usr/lib/perl5/vendor_perl/5.8.1 /usr/lib/perl5/vendor_perl .

Environment for perl v5.8.8: HOME=/home/pixel LANG (unset) LANGUAGE (unset) LC_SOURCED=1 LD_LIBRARY_PATH=/home/pixel/lib/i386:/usr/X11R6/lib:/usr/X11R6/lib/modules LOGDIR (unset) PATH=/home/pixel/bin:/home/pixel/bin/oneliner:/home/pixel/bin/perl:/home/pixel/bin/ruby:/home/pixel/bin/python:/home/pixel/bin/tcl:/home/pixel/bin/i386:/usr/freeware/bin:/usr/etc:/usr/local/bin:/usr/X11R6/bin:/sbin:/usr/sbin:/sbin:/usr/sbin:/usr/X11R6/bin:/bin:/usr/bin:/usr/local/bin:/root/bin:/root/bin:/usr/X11R6/bin PERL5LIB=/home/pixel/etc/perl:/home/pixel/etc/perl/i386-linux:/home/pixel/etc/perl/site_perl PERL_BADLANG (unset) SHELL=/bin/bash

p5pRT commented 17 years ago

From @eserte

"pixel@mandriva.com (via RT)" \perlbug\-followup@perl\.org writes:

# New Ticket Created by pixel@mandriva.com # Please include the string: [perl #41442] # in the subject line of all future correspondence about this issue. # \<URL: http://rt.perl.org/rt3/Ticket/Display.html?id=41442 >

This is a bug report for perl from pixel@mandriva.com\, generated with the help of perlbug 1.35 running under perl v5.8.8.

this segfaults if the locale charset is not utf8:

use open ':locale'; print STDERR "\x{201e}"; # „

This seems to be an endless loop. This part is repeated over and over again (tried with bleedperl):

#852 0x282f9fc8 in PerlIOEncode_flush (f=0x81c0014) at encoding.xs:419 #853 0x0813b79c in Perl_PerlIO_flush (f=0x81c0014) at perlio.c:1655 #854 0x0813eba8 in PerlIOBuf_write (f=0x81c0014\, vbuf=0x188f25c8\, count=39) at perlio.c:3927 #855 0x282fb9de in PerlIOEncode_write (f=0x81c0014\, vbuf=0x188f25c8\, count=39) at encoding.xs:582 #856 0x0813b64a in Perl_PerlIO_write (f=0x81c0014\, vbuf=0x188f25c8\, count=39) at perlio.c:1631 #857 0x0809a7fe in Perl_write_to_stderr ( message=0x188f25c8 "\"\\x{201e}\" does not map to iso-8859-1.\n"\, msglen=39) at util.c:1211 #858 0x0809b16e in Perl_vwarn ( pat=0x282f24a4 "\"\\x{%04lx}\" does not map to %s"\, args=0xbbc103c4) at util.c:1431 ---Type \ to continue\, or q \ to quit--- #859 0x0809b44b in Perl_vwarner (err=44\, pat=0x282f24a4 "\"\\x{%04lx}\" does not map to %s"\, args=0xbbc103c4) at util.c:1508 #860 0x0809b1b2 in Perl_warner (err=44\, pat=0x282f24a4 "\"\\x{%04lx}\" does not map to %s") at util.c:1481 #861 0x282eb060 in encode_method (enc=0x282f3160\, dir=0x282f30e0\, src=0x81a71c8\, check=2306\, offset=0x0\, term=0x0\, retcode=0x0) at Encode.xs:187 #862 0x282ee82a in XS_Encode__XS_encode (cv=0x825b6a8) at Encode.xs:620 #863 0x080c13b3 in Perl_pp_entersub () at pp_hot.c:2802 #864 0x0809859a in Perl_runops_debug () at dump.c:1902 #865 0x080b252f in Perl_call_sv (sv=0x188fa128\, flags=64) at perl.c:2647 #866 0x080b22d3 in Perl_call_method (methname=0x282fcb08 "encode"\, flags=0) at perl.c:2580

Regards\, Slaven

--- Flags: category=core severity=medium --- Site configuration information for perl v5.8.8:

Configured by Mandriva at Fri Sep 8 20:00:54 CEST 2006.

Summary of my perl5 (revision 5 version 8 subversion 8) configuration: Platform: osname=linux\, osvers=2.6.12-12mdksmp\, archname=i386-linux uname='linux n4.mandriva.com 2.6.12-12mdksmp #1 smp fri sep 9 17:43:23 cest 2005 i686 intel(r) xeon(tm) cpu 2.80ghz gnulinux ' config_args='-des -Dinc_version_list=5.8.7 5.8.7/i386-linux 5.8.6 5.8.6/i386-linux 5.8.5 5.8.4 5.8.3 5.8.2 5.8.1 5.8.0 5.6.1 5.6.0 -Darchname=i386-linux -Dcc=gcc -Doptimize=-O2 -pipe -Wp\,-D_FORTIFY_SOURCE=2 -fexceptions -fomit-frame-pointer -march=i586 -mtune=pentiumpro -fasynchronous-unwind-tables -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr -Dsitebin=/usr/local/bin -Dsiteman1dir=/usr/local/share/man/man1 -Dsiteman3dir=/usr/local/share/man/man3 -Dman3ext=3pm -Dcf_by=Mandriva -Dmyhostname=localhost -Dperladmin=root@localhost -Dcf_email=root@localhost -Dd_dosuid -Ud_csh -Duseshrplib' hint=recommended\, useposix=true\, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='gcc'\, ccflags ='-fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm'\, optimize='-O2 -pipe -Wp\,-D_FORTIFY_SOURCE=2 -fexceptions -fomit-frame-pointer -march=i586 -mtune=pentiumpro -fasynchronous-unwind-tables'\, cppflags='-fno-strict-aliasing -pipe -Wdeclaration-after-statement -I/usr/local/include -I/usr/include/gdbm' ccversion=''\, gccversion='4.1.1 20060724 (prerelease) (4.1.1-3mdk)'\, gccosandvers='' intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=12 ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=4\, prototype=define Linker and Libraries: ld='gcc'\, ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lnsl -lndbm -lgdbm -ldl -lm -lcrypt -lutil -lc perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc libc=/lib/libc-2.4.so\, so=so\, useshrplib=true\, libperl=libperl.so gnulibc_version='2.4' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E -Wl\,-rpath\,/usr/lib/perl5/5.8.8/i386-linux/CORE' cccdlflags='-fPIC'\, lddlflags='-shared -L/usr/local/lib'

Locally applied patches: Mandriva Linux patches

--- @INC for perl v5.8.8: /home/pixel/etc/perl /home/pixel/etc/perl/i386-linux /home/pixel/etc/perl/site_perl /usr/lib/perl5/5.8.8/i386-linux /usr/lib/perl5/5.8.8 /usr/lib/perl5/site_perl/5.8.8/i386-linux /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.8/i386-linux /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl/5.8.7 /usr/lib/perl5/vendor_perl/5.8.7/i386-linux /usr/lib/perl5/vendor_perl/5.8.6 /usr/lib/perl5/vendor_perl/5.8.6/i386-linux /usr/lib/perl5/vendor_perl/5.8.5 /usr/lib/perl5/vendor_perl/5.8.4 /usr/lib/perl5/vendor_perl/5.8.3 /usr/lib/perl5/vendor_perl/5.8.2 /usr/lib/perl5/vendor_perl/5.8.1 /usr/lib/perl5/vendor_perl .

--- Environment for perl v5.8.8: HOME=/home/pixel LANG (unset) LANGUAGE (unset) LC_SOURCED=1 LD_LIBRARY_PATH=/home/pixel/lib/i386:/usr/X11R6/lib:/usr/X11R6/lib/modules LOGDIR (unset) PATH=/home/pixel/bin:/home/pixel/bin/oneliner:/home/pixel/bin/perl:/home/pixel/bin/ruby:/home/pixel/bin/python:/home/pixel/bin/tcl:/home/pixel/bin/i386:/usr/freeware/bin:/usr/etc:/usr/local/bin:/usr/X11R6/bin:/sbin:/usr/sbin:/sbin:/usr/sbin:/usr/X11R6/bin:/bin:/usr/bin:/usr/local/bin:/root/bin:/root/bin:/usr/X11R6/bin PERL5LIB=/home/pixel/etc/perl:/home/pixel/etc/perl/i386-linux:/home/pixel/etc/perl/site_perl PERL_BADLANG (unset) SHELL=/bin/bash

-- Slaven Rezic - slaven \ rezic \ de babybike - routeplanner for cyclists in Berlin handheld (e.g. Compaq iPAQ with Linux) version of bbbike http://bbbike.sourceforge.net

p5pRT commented 17 years ago

The RT System itself - Status changed from 'new' to 'open'

p5pRT commented 17 years ago

From @eserte

"pixel@mandriva.com (via RT)" \perlbug\-followup@perl\.org writes:

# New Ticket Created by pixel@mandriva.com # Please include the string: [perl #41442] # in the subject line of all future correspondence about this issue. # \<URL: http://rt.perl.org/rt3/Ticket/Display.html?id=41442 >

This is a bug report for perl from pixel@mandriva.com\, generated with the help of perlbug 1.35 running under perl v5.8.8.

this segfaults if the locale charset is not utf8:

use open ':locale'; print STDERR "\x{201e}"; # „

Attached is a patch which provides a workaround for the problem by trapping recursive calls to the encode() method in PerlIO/encoding/encoding.xs. However\, I am far from sure that this is solved properly.

p5pRT commented 17 years ago

From @eserte

perlio-encoding.patch

```diff # # # To apply this patch: # STEP 1: Chdir to the source directory. # STEP 2: Run the 'applypatch' program with this patch file as input. # # If you do not have 'applypatch', it is part of the 'makepatch' package # that you can fetch from the Comprehensive Perl Archive Network: # http://www.perl.com/CPAN/authors/Johan_Vromans/makepatch-x.y.tar.gz # In the above URL, 'x' should be 2 or higher. # # To apply this patch without the use of 'applypatch': # STEP 1: Chdir to the source directory. # STEP 2: Run the 'patch' program with this file as input. # #### End of Preamble #### #### Patch data follows #### diff -up 'bleedperl/ext/PerlIO/encoding/encoding.xs' 'bleedperl3/ext/PerlIO/encoding/encoding.xs' Index: ./ext/PerlIO/encoding/encoding.xs Prereq: 0.3 --- ./ext/PerlIO/encoding/encoding.xs Tue Jun 13 21:29:10 2006 +++ ./ext/PerlIO/encoding/encoding.xs Mon Feb 5 22:41:59 2007 @@ -48,6 +48,7 @@ typedef struct { SV *enc; /* the encoding object */ SV *chk; /* CHECK in Encode methods */ int flags; /* Flags currently just needs lines */ + int inEncodeCall; /* trap recursive encode calls */ } PerlIOEncode; #define NEEDS_LINES 1 @@ -147,6 +148,7 @@ PerlIOEncode_pushed(pTHX_ PerlIO * f, co } e->chk = newSVsv(get_sv("PerlIO::encoding::fallback", 0)); + e->inEncodeCall = 0; FREETMPS; LEAVE; @@ -404,6 +406,7 @@ PerlIOEncode_flush(pTHX_ PerlIO * f) STRLEN len; SSize_t count = 0; if ((PerlIOBase(f)->flags & PERLIO_F_WRBUF) && (e->base.ptr > e->base.buf)) { + if (e->inEncodeCall) return 0; /* Write case - encode the buffer and write() to layer below */ PUSHSTACKi(PERLSI_MAGIC); SPAGAIN; @@ -416,9 +419,12 @@ PerlIOEncode_flush(pTHX_ PerlIO * f) XPUSHs(e->bufsv); XPUSHs(e->chk); PUTBACK; + e->inEncodeCall = 1; if (call_method("encode", G_SCALAR) != 1) { + e->inEncodeCall = 0; Perl_die(aTHX_ "panic: encode did not return a value"); } + e->inEncodeCall = 0; SPAGAIN; str = POPs; PUTBACK; @@ -453,6 +459,7 @@ PerlIOEncode_flush(pTHX_ PerlIO * f) } /* See if there is anything left in the buffer */ if (e->base.ptr < e->base.end) { + if (e->inEncodeCall) return 0; /* Bother - have unread data. re-encode and unread() to layer below */ @@ -472,9 +479,12 @@ PerlIOEncode_flush(pTHX_ PerlIO * f) XPUSHs(str); XPUSHs(e->chk); PUTBACK; + e->inEncodeCall = 1; if (call_method("encode", G_SCALAR) != 1) { - Perl_die(aTHX_ "panic: encode did not return a value"); + e->inEncodeCall = 0; + Perl_die(aTHX_ "panic: encode did not return a value"); } + e->inEncodeCall = 0; SPAGAIN; str = POPs; PUTBACK; #### End of Patch data #### #### ApplyPatch data follows #### # Data version : 1.0 # Date generated : Mon Feb 5 22:58:23 2007 # Generated by : makepatch 2.03 # Recurse directories : Yes # Excluded files : (\A|/).*\~\Z # (\A|/).*\.a\Z # (\A|/).*\.bak\Z # (\A|/).*\.BAK\Z # (\A|/).*\.elc\Z # (\A|/).*\.exe\Z # (\A|/).*\.gz\Z # (\A|/).*\.ln\Z # (\A|/).*\.o\Z # (\A|/).*\.obj\Z # (\A|/).*\.olb\Z # (\A|/).*\.old\Z # (\A|/).*\.orig\Z # (\A|/).*\.rej\Z # (\A|/).*\.so\Z # (\A|/).*\.Z\Z # (\A|/)\.del\-.*\Z # (\A|/)\.make\.state\Z # (\A|/)\.nse_depinfo\Z # (\A|/)core\Z # (\A|/)tags\Z # (\A|/)TAGS\Z # v 'patchlevel.h' 4732 1160388693 33056 # p 'ext/PerlIO/encoding/encoding.xs' 16562 1170711719 0100440 #### End of ApplyPatch data #### #### End of Patch kit [created: Mon Feb 5 22:58:23 2007] #### #### Patch checksum: 99 3441 30793 #### #### Checksum: 117 4065 16793 #### ```

p5pRT commented 17 years ago

From @eserte

-- Slaven Rezic - slaven \ rezic \ de

tkrevdiff - graphical display of diffs between revisions (RCS\, CVS or SVN) http://ptktools.sourceforge.net/#tkrevdiff

p5pRT commented 17 years ago

From @rgarcia

On 05 Feb 2007 23:04:07 +0100\, Slaven Rezic \slaven@rezic\.de wrote:

use open ':locale'; print STDERR "\x{201e}"; # „

Attached is a patch which provides a workaround for the problem by trapping recursive calls to the encode() method in PerlIO/encoding/encoding.xs. However\, I am far from sure that this is solved properly.

The patch might be interesting to prevent similar segfaults in the future\, but it doesn't fix the problem. I haven't looked at the code yet\, but it seems that Encode tries to print a warning\, but STDERR being already in use\, somehow that clashes. Maybe disable re-encoding when printing warnings ?

p5pRT commented 17 years ago

From @eserte

"Rafael Garcia-Suarez" \rgarciasuarez@gmail\.com writes:

On 05 Feb 2007 23:04:07 +0100\, Slaven Rezic \slaven@rezic\.de wrote:

use open ':locale'; print STDERR "\x{201e}"; # „

Attached is a patch which provides a workaround for the problem by trapping recursive calls to the encode() method in PerlIO/encoding/encoding.xs. However\, I am far from sure that this is solved properly.

The patch might be interesting to prevent similar segfaults in the future\, but it doesn't fix the problem. I haven't looked at the code yet\, but it seems that Encode tries to print a warning\, but STDERR being already in use\, somehow that clashes. Maybe disable re-encoding when printing warnings ?

Yes\, this is the problem\, and this is what the patch tries to fix. The problematic path looks as following: PerlIOEncode_flush is called while there are some (utf-8) bytes in the e->bufsv buffer (which seems to be the same content like e->base.buf\, as it looks from a gdb session). Then encode() is called with this buffer\, detects that the character in the buffer does not fit into iso-8859-1 (or whatever target encoding is causing the problem) and issues a warning. This warning will be written to STDERR (again!)\, meaning the warning string is appended to the above mentioned buffer. Eventually PerlIOEncode_flush is called again\, still with the problem-causing bytes at the beginning\, which would cause another warning and so on\, until all memory is eaten up.

With the inEncodeCall flag I tried to prevent calling encode() again recursively.

I am not sure if the patch is right\, because * Is it OK to add another element to the PerlIOEncode struct (because of binary compatibility or so?)= * Is it guaranteed that the call to the encode() method will always return? Otherwise there could be a pending inEncodeCall flag never reset again. * Are there legitimate cases where there could be recursive encode() calls?

Regards\, Slaven

-- Slaven Rezic - slaven \ rezic \ de

tksm - Perl/Tk program for searching and replacing in multiple files http://ptktools.sourceforge.net/#tksm

p5pRT commented 17 years ago

From @rgarcia

On 06 Feb 2007 21:59:53 +0100\, Slaven Rezic \slaven@rezic\.de wrote:

Yes\, this is the problem\, and this is what the patch tries to fix. The problematic path looks as following: PerlIOEncode_flush is called while there are some (utf-8) bytes in the e->bufsv buffer (which seems to be the same content like e->base.buf\, as it looks from a gdb session). Then encode() is called with this buffer\, detects that the character in the buffer does not fit into iso-8859-1 (or whatever target encoding is causing the problem) and issues a warning. This warning will be written to STDERR (again!)\, meaning the warning string is appended to the above mentioned buffer. Eventually PerlIOEncode_flush is called again\, still with the problem-causing bytes at the beginning\, which would cause another warning and so on\, until all memory is eaten up.

With the inEncodeCall flag I tried to prevent calling encode() again recursively.

Thanks\, applied as change #30213\, adding also Pixel's test case. (and bumping $VERSION) Certainly not as elegant as having a fully reentrant thing\, but works.

I am not sure if the patch is right\, because * Is it OK to add another element to the PerlIOEncode struct (because of binary compatibility or so?)=

I don't see why it wouldn't be.

* Is it guaranteed that the call to the encode() method will always return? Otherwise there could be a pending inEncodeCall flag never reset again. * Are there legitimate cases where there could be recursive encode() calls?

I don't think so\, since that wasn't designed for that apparently.

p5pRT commented 17 years ago

@rgs - Status changed from 'open' to 'resolved'

p5pRT commented 8 years ago

From @dcollinsn

This was fixed somewhere between 5.8.8 and 5.8.9 by this monster of a commit:

f266b493bceb33a30e1a797a79465aa4e1fca744 is the first bad commit commit f266b493bceb33a30e1a797a79465aa4e1fca744 Author: Nicholas Clark \nick@ccl4\.org Date: Sat Feb 17 17:02:53 2007 +0000

Integrate: [ 28629] Subject: [PATCH] z/OS: an easy test nit From: Jarkko Hietaniemi \jhi@cc\.hut\.fi Date: Thu\, 27 Jul 2006 20:00:02 +0300 (EEST) Message-Id: \200607271700\.k6RH02V1355005@kosh\.hut\.fi

[ 28766] Fix PerlIO::scalar fileno() documentation as suggested in: Subject: [perl #40245] POD error for PerlIO::scalar in 5.8.8 From: "Shawn Boyette" (via RT) \perlbug\-followup@perl\.org Date: Mon\, 28 Aug 2006 00:21:25 -0700 Message-ID: \rt\-3\.5\.HEAD\-31263\-1156749685\-634\.40245\-75\-0@perl\.org

[ 28798] Fix for [perl #40267] PerlIO::scalar doesn't respect readonly-ness

[ 28799] Bump version of PerlIO::scalar

[ 28903] Subject: [PATCH] PerlIO::scalar (aka open(my $fh\, >\$foo)): zero-filling seekand don't talk to negative strangers From: Jarkko Hietaniemi \jhi@iki\.fi Date: Fri\, 29 Sep 2006 17:41:28 +0300 Message-ID: \451D3098\.1000305@iki\.fi

[ 29173] Doc clarification for PerlIO::encoding (thanks to Steve Hay)

[ 29702] Bump version of PerlIO::via after last change

[ 29751] Fix bug #40407: after a seek on a PerlIO::scalar filehandle\, ensure there's a string buffer in the scalar

[ 30213] Subject: Re: [perl #41442] segfault (dead loop) with Encoding\, use open :locale\, print STDERR From: Slaven Rezic \slaven@rezic\.de Date: 05 Feb 2007 23:04:07 +0100 Message-ID: \87veiggt2g\.fsf@biokovo\.herceg\.de

[ 30214] Explicitly load PerlIO::encoding when testing it. (so we know what we're testing)

[ 30233] Skip this test if "use open" fails due to an unknown encoding p4raw-link: @30233 on //depot/perl: 771ed9f1af61a5ebf694942246ba6240a0a817ce p4raw-link: @30214 on //depot/perl: af65d5cd78551c53080e83951660cf3df8d4d4e9 p4raw-link: @30213 on //depot/perl: 74f6c1ca58b1c40741f55591ab97a77b6751f510 p4raw-link: @29751 on //depot/perl: 8b8eea963147e9111f99b4c1bf3eb8b60fe51935 p4raw-link: @29702 on //depot/perl: c02d4329fd236df2f6781ece55e9312b6f67bc10 p4raw-link: @29173 on //depot/perl: 51dfe3f7e03fd8d2e9dadd18c223dce6ad08480f p4raw-link: @28903 on //depot/perl: 42bc49da149640802c6d82e088ba670810f22d2a p4raw-link: @28799 on //depot/perl: ad95c6e182499b977b9e82cd165b06b867730861 p4raw-link: @28798 on //depot/perl: b35bc0c6c55657527d346e6d05f46024491a31b0 p4raw-link: @28766 on //depot/perl: 846c5ed3bd6bbf47ed6f029fd3595992cfcdf1c7 p4raw-link: @28629 on //depot/perl: 501f55b944ed34544942c9ed775ce8fe98a3fa2e

p4raw-id: //depot/maint-5.8/perl@30340 p4raw-branched: from //depot/perl@30339 'branch in' ext/PerlIO/encoding/t/nolooping.t (@30214..) p4raw-integrated: from //depot/perl@30339 'copy in' ext/PerlIO/t/encoding.t (@23631..) ext/PerlIO/encoding/encoding.xs (@26175..) ext/PerlIO/via/via.pm (@26817..) ext/PerlIO/t/scalar.t (@28798..) ext/PerlIO/encoding/encoding.pm (@29173..) p4raw-integrated: from //depot/perl@30213 'merge in' MANIFEST (@30211..) p4raw-integrated: from //depot/perl@28903 'merge in' pod/perldiag.pod (@28868..) p4raw-integrated: from //depot/perl@28766 'ignore' ext/PerlIO/scalar/scalar.pm (@24543..) p4raw-integrated: from //depot/perl@24271 'ignore' ext/PerlIO/scalar/scalar.xs (@24248..) 'merge in' ext/PerlIO/via/via.xs (@20614..)

:100644 100644 fb9a35503bbd64f925752279f2d4d92ae4aec465 4b59564147fb0caec127a3dcdccdca435cf74027 M MANIFEST :040000 040000 6c92bbfba286ee95cbb9282bab34e40a724e55be f019c4bfb1363d16d93de050f97b5f996554f031 M ext :040000 040000 2d3343bf4eed37ea727bcba33f63c07c4629be3f cb0017f3a3af04895113627e09f4e8f7d30b5dfc M pod bisect run success That took 1710 seconds.

Test to follow.

-- Respectfully\, Dan Collins

p5pRT commented 8 years ago

From @dcollinsn

This (RT #37234) is\, in fact\, a duplicate of RT #41442. 41442 /has/ a test\, and their test is better than my test. So\, merging and closing.

-- Respectfully\, Dan Collins

p5pRT commented 8 years ago

@dcollinsn - Status changed from 'open' to 'resolved'

Perl / perl5